Memristor apparatus

ABSTRACT

A memristor apparatus includes meta-stable switching elements and an AHAH (Anti-Hebbian and Hebbian) feedback mechanism that operates the meta-stable switching elements.

CROSS-REFERENCE TO PATENT APPLICATIONS

This patent application is a continuation of U.S. patent applicationSer. No. 13/421,398 entitled “Memristor Apparatus,” filed on Mar. 15,2012, which is a continuation-in-part of U.S. patent application Ser.No. 12/100,586 entitled “Adaptive Neural Network UtilizingNanotechnology-Based Components,” filed on Apr. 10, 2008, which is acontinuation of U.S. patent application Ser. No. 10/969,789, filed onOct. 21, 2004, which is a Continuation-in-Part of U.S. patentapplication Ser. No. 10/730,708, entitled “Adaptive Neural NetworkUtilizing Nanotechnology-Based Components,” filed on Dec. 8, 2003, whichclaims priority to U.S. Provisional Patent Application Ser. No.60/458,024 filed on Mar. 27, 2003. U.S. patent application Ser. No.10/969,789 is a continuation-in-part of U.S. patent application Ser. No.10/095,273, “Physical Neural Network Design IncorporatingNanotechnology,” which was filed on Mar. 12, 2002, the disclosure ofwhich is incorporated herein by reference. U.S. patent application Ser.No. 10/969,789 is also a continuation-in-part of U.S. patent applicationSer. No. 10/162,524, “Multi-Layer Training in a Physical Neural NetworkFormed Utilizing Nanotechnology,” which was filed on Jun. 5, 2002, thedisclosure of which is incorporated herein by reference. U.S. patentapplication Ser. No. 10/969,789 is additionally a continuation-in-partof U.S. patent application Ser. No. 10/226,191, “High-Density SynapseChip Using Nanoparticles,” which was filed on Aug. 22, 2002, thedisclosure of which is incorporated herein by reference. U.S. patentapplication Ser. No. 10/969,789 is also a continuation-in-part of U.S.patent application Ser. No. 10/748,546, “Physical Neural Network LiquidState Machine Utilizing Nanotechnology,” which was filed on Dec. 30,2003, the disclosure of which is incorporated herein by reference. U.S.patent application Ser. No. 10/969,789 is also a continuation-in-part ofU.S. patent application Ser. No. 10/748,631, “Application of Hebbian andAnti-Hebbian Learning to Nanotechnology-Based Physical Neural Networks,”which was filed on Dec. 30, 2003, the disclosure of which isincorporated herein by reference. U.S. patent application Ser. No.10/969,789 is also a continuation-in-part of U.S. patent applicationSer. No. 10/730,708, “Adaptive Neural Network UtilizingNanotechnology-Based Components,” which was filed on Dec. 8, 2003, thedisclosure of which is incorporated herein by reference. Thisapplication therefore traces and claims priority to the earliestpriority date of U.S. patent application Ser. No. 10/095,273, “PhysicalNeural Network Design Incorporating Nanotechnology,” which was filed onMar. 12, 2002.

U.S. patent application Ser. No. 13/421,398 is also acontinuation-in-part of U.S. patent application Ser. No. 12/612,677,filed on Nov. 5, 2009. U.S. patent application Ser. No. 12/612,677 isincorporated herein by reference in its entirety. U.S. patentapplication Ser. No. 13/421,398 is also a continuation-in-part of U.S.patent application Ser. No. 12/938,537, which was filed on Nov. 3, 2010.U.S. patent application Ser. No. 12/938,537 is incorporated herein byreference in its entirety. U.S. patent application Ser. No. 13/421,398is also a continuation-in-part of U.S. patent application Ser. No.12/974,829, which was filed on Dec. 21, 2010. U.S. patent applicationSer. No. 12/974,829 is additionally incorporated herein by reference inits entirety. U.S. patent application Ser. No. 13/421,398 isadditionally a continuation-in-part of U.S. patent application Ser. No.13/113,167, which was filed on May 23, 2011. U.S. patent applicationSer. No. 13/113,167 is incorporated by reference herein in its entirety.U.S. patent application Ser. No. 13/421,398 is also acontinuation-in-part of U.S. patent application Ser. No. 13/354,537,which was filed on Jan. 20, 2012. U.S. patent application Ser. No.13/354,537 is incorporated by reference herein in its entirety. U.S.patent application Ser. No. 13/421,398 is also a continuation-in-part ofU.S. patent application Ser. No. 13/268,119, which was filed on Oct. 7,2011. U.S. patent application Ser. No. 13/268,119 is also hereinincorporated by reference in its entirety.

TECHNICAL FIELD

Embodiments generally relate to the field of thermodynamic computing.Embodiments additionally relate to memristor devices and logic gatecomponents constructed that incorporate the use of unstable switchingelements. Embodiments additionally relate to self-assembling andrepairing methods and systems. Embodiments also relate to nanoscalemeta-stable devices, such as Knowm™ synapses, switching molecules andcross bar switching architectures.

BACKGROUND

A memristor, also sometimes referred to as a “memory resistor” is apassive two-terminal electrical component that can function as anon-linear circuit element relating charge and magnetic flux linkage.When current flows in one direction through the device, the electricalresistance can increase. When current flows in the opposite direction,the resistance can decrease. When the current is stopped, the deviceretains the last resistance that it had, and when the flow of chargestarts again, the resistance of the circuit will be what it was when itwas last active. A memristor thus has a regime of operation with anapproximately linear charge-resistance relationship as long as thetime-integral of the current stays within certain bounds.

Devices based on thermodynamic computing have been implemented. Oneexample of such a device is the Knowm™ network or system, which isdescribed in a number of U.S. patents and publications. U.S. Pat. No.6,889,216, entitled “Physical Neural Network Design IncorporatingNanotechnology,” which issued to Alex Nugent on May 3, 2005 generallydescribes a physical neural network that generally includes one or moreneuron-like nodes, which are formed from a plurality of interconnectednanoconnections formed from nanoconductors. Such connections constituteKnowm™ connections. Each neuron-like node sums one or more input signalsand generates one or more output signals based on a threshold associatedwith the input signal.

The Knowm™ device physical neural network also includes a connectionnetwork formed from the interconnected nanoconnections, such that theinterconnected nanoconnections used thereof by one or more of theneuron-like nodes are strengthened or weakened according to anapplication of an electric field, variations in frequency, and so forth.U.S. Pat. No. 6,889,216 is incorporated herein by reference.

Another example of a Knowm™ network or system is described in U.S.Patent Publication No. 20030236760, entitled “Multi-layer Training in aPhysical Neural Network Formed Utilizing Nanotechnology,” by inventorAlex Nugent, which was published on Dec. 25, 2003. U.S. PatentPublication No. 20030236760 generally describes methods and systems fortraining at least one connection network located between neuron layerswithin a multi-layer physical neural network (e.g., a Knowm™ network ordevice). The multi-layer physical neural network described in U.S.Patent Publication No. 20030236760 can be formed with a plurality ofinputs and a plurality outputs thereof, wherein the multi-layer physicalneural network comprises a plurality of layers therein, such that eachlayer thereof comprises at least one connection network and at least oneassociated neuron.

Thereafter, a training wave, as further described in U.S. PatentPublication No. 20030236760, can be initiated across one or moreconnection networks associated with an initial layer of the multi-layerphysical neural network which propagates thereafter through succeedingconnection networks of succeeding layers of the multi-layer physicalneural network by successively closing and opening at least one switchassociated with each layer of the multi-layer physical neural network.At least one feedback signal thereof can be automatically provided toeach preceding connection network associated with each preceding layerthereof to strengthen or weaken nanoconnections associated with eachconnection network of the multi-layer physical neural network. U.S.Patent Publication No. 20030236760 is incorporated herein by reference.

A further example of a Knowm™ network or system is described in U.S.Patent Publication No. 20040039717, entitled High-density synapse chipusing nanoparticles” by inventor Alex Nugent. U.S. Patent PublicationNo. 20040039717 published on Feb. 26, 2004 and generally describes aphysical neural network synapse chip (i.e., a Knowm™ chip) and a methodfor forming such a synapse chip. The synapse or Knowm™ chip can beconfigured to include an input layer comprising a plurality of inputelectrodes and an output layer comprising a plurality of outputelectrodes, such that the output electrodes are located perpendicular tothe input electrodes. A gap is generally formed between the input layerand the output layer.

A solution can then be provided which is prepared from a plurality ofnanoconductors and a dielectric solvent. The solution is located withinthe gap, such that an electric field is applied across the gap from theinput layer to the output layer to form nanoconnections of a physicalneural network implemented by the synapse chip. Such a gap can thus beconfigured as an electrode gap. The input electrodes can be configuredas an array of input electrodes, while the output electrodes can beconfigured as an array of output electrodes. U.S. Patent Publication No.20040039717 is also incorporated herein by reference.

A further example of a Knowm™ network or system is disclosed in U.S.Patent Publication No. 20040153426, entitled “Physical Neural NetworkLiquid State Machine Utilizing Nanotechnology,” by inventor Alex Nugent,which was published on Aug. 5, 2004. U.S. Patent Publication No.20040153426 generally discloses a physical neural network (i.e., aKnowm™ network), which functions as a liquid state machine.

The physical neural network described in U.S. Patent Publication No.20040153426 can be configured from molecular connections located withina dielectric solvent between pre-synaptic and post-synaptic electrodesthereof, such that the molecular connections are strengthened orweakened according to an application of an electric field or a frequencythereof to provide physical neural network connections thereof. Asupervised learning mechanism is associated with the liquid statemachine, whereby connections strengths of the molecular connections aredetermined by pre-synaptic and post-synaptic activity respectivelyassociated with the pre-synaptic and post-synaptic electrodes, whereinthe liquid state machine comprises a dynamic fading memory mechanism.U.S. Patent Publication No. 20040153426 is also incorporated herein byreference.

A further example of a Knowm™ network or system is disclosed in U.S.Patent Publication No. 20040162796, entitled “Application of Hebbian andanti-Hebbian Learning to Nanotechnology-based Physical Neural Networks”by inventor Alex Nugent, which published on Aug. 19, 2004. U.S. PatentPublication No. 20040162796 generally discloses a physical neuralnetwork (i.e., Knowm™ network) configured utilizing nanotechnology. TheKnowm™ network disclosed in U.S. Patent Publication No. 20040162796includes a plurality of molecular conductors (e.g., nanoconductors)which form neural connections between pre-synaptic and post-synapticcomponents of the physical neural network.

An alternative example of a nanoscale meta-stable switching element isthe cross bar architecture. A molecular cross bar memory is disclosed,for example, in U.S. Pat. No. 6,128,214 entitled “Molecular Wire Crossbar Memory” which issued to Kuekes et al. on Oct. 3, 2000. Kuekes et aldisclose a memory device that is constructed from cross bar arrays ofnanowires sandwiching molecules that act as on/off switches. The deviceis formed from a plurality of nanometer-scale devices, each devicecomprising a junction formed by a pair of crossed wires where one wirecrosses another and at least one connector species connects the pair ofcrossed wires in the junction. The connector species comprises abi-stable molecular switch. The junction forms either a resistor or adiode or an asymmetric non-linear resistor. The junction possesses astate that is capable of being altered by application of a first voltageand sensed by the application of a second, non-destructive voltage. Aseries of related technology attempts to convert everything frommolecular logic to how to chemically assemble these devices.

Such a molecular cross bar device has two general applications. Thenotion of transistors built from nanotubes and relying on nanotubesproperties is being pursued. In this manner, computational systems canbe constructed. Second, two wires can be selectively brought to acertain voltage and the resulting electrostatic force attracts them.When the wires touch, the Van der Waals force maintains the wires incontact with one another such that a “bit” is stored. The connections insuch an apparatus can therefore be utilized with standard electroniccircuitry.

One aspect of the cross bar architecture, which deserves someillumination, is its potential unreliability. The device functions bycreating a physical system with one meta-stable state, which is formedfrom the balance of van der-walls intermolecular attraction and amechanical tension from a bent nanowire. The van der-walls force must bemade sufficient to oppose the mechanical strain. The construction ofsuch a device preferably utilizes a nanowire suspended above a lowernanotube (or other nanoparticle) or electrical contact. The furtherapart the supports, the less of a force required to bend the nanowire.This has an interesting consequence when the system in scaled down.

As the switch density is increased, the support distance mustnecessarily decrease. This causes the force from mechanical strain toincrease. One possible solution would be to place the nanowires closertogether, so that less of a deflection is needed to make contact. Thereis an absolute distance, however, for which the nanowires cannot bebrought closer, and this distance is set by quantum mechanical electrontunneling. In this manner, it can be seen that as the device is scaleddown, the potential energy well formed from the Van der Waals forcebecomes weakened as the mechanical strain from the bent nanowireincreases. This results in the connection having an increasedprobability of falling into the “ground state”.

Alternately, switching molecules have been found that can be configuredto be in a conducting and a non-conducting state. To date, one problemwith such molecules is that the states are only meta-stable,particularly one of the states. After a short time, which usuallydecreases as the temperature is increased, the state is lost. If the“on” state represents a conducting state, then this naturally raises thequestion, “how does one work with an unreliable connection?” It is thepurpose of this disclosure to apply a type of local interaction so as towork with an unreliable connection, or alternately to constantly repaira connection if it undergoes a transition from a meta-stable to a groundstate. This methodology can be used in, for example, Knowm™ connectionnetworks, nanowire cross junctions and meta-stable switching moleculessuch as rotoxane. The methodology has been first described for Knowm™connection networks, and its extension to other nanoscaleimplementations is unexpected. In all cases, we can refer to pre- andpost-synaptic electrodes, where a Knowm™ connection, nanowire junction,switching molecule, quantum tunneling transistor and the like forms thepre-to-post electrode switching contact. In all cases, the device isconfigured to work with a meta-stable nanoscale switch, which we willdescribe in detail and offer examples.

A local feedback mechanism can be applied, which implements Hebbian andanti-Hebbian learning. Such a learning mechanism can utilize a voltagegradient or voltage gradient dependencies to implement Hebbian and/oranti-Hebbian (AHAH) plasticity. The learning mechanism can also utilizepre-synaptic and post-synaptic frequencies to provide Hebbian and/oranti-Hebbian learning within the physical neural network. U.S. PatentPublication No. 20040162796 is incorporated herein by reference.

An additional example of a Knowm™ network or device is disclosed in U.S.Patent Publication No. 20040193558, entitled “Adaptive Neural NetworkUtilizing Nanotechnology-based Components” by Alex Nugent, whichpublished on Sep. 30, 2004. U.S. Patent Publication No. 20040193558generally describes methods and systems for modifying at least onesynapse of a physical neural network (i.e., a Knowm™ network). Thephysical neural or Knowm™ network described in U.S. Patent PublicationNo. 20040193558 can be implemented as an adaptive neural network, whichincludes one or more neurons and one or more synapses thereof.

The synapses are formed from a plurality of nanoparticles disposedwithin a dielectric solution in association with one or morepre-synaptic electrodes and one or more post-synaptic electrodes and anapplied electric field. Alternately, the synapses are formed from aplurality of nanowire cross junctions formed from two or more nanowirescomposed of, for example, carbon nanotubes and forming one or morepre-synaptic electrodes and one or more post-synaptic electrodes.Alternately, The synapses are formed from a plurality of switchingmolecules formed from one or more molecule of, for example, Rotoxane,preferably located between two or more pre- and post-synaptic electrodescomprising, for example, carbon nanotubes or photolithography definedand patterned electrodes.

At least one electric pulse can be generated from one or more of theneural circuit modules to one or more of the pre-synaptic electrodes ofa succeeding neuron and one or more post-synaptic electrodes of one ormore of the neurons of the synapse junction, thereby strengthening atleast one nanoparticle of a plurality of nanoparticles disposed withinthe dielectric solution and at least one synapse thereof. At least oneelectric pulse can be generated from one or more of the neural circuitmodules to one or more of the pre-synaptic electrodes of a succeedingneuron and one or more post-synaptic electrodes of one or more of theneurons of the synapse junction, thereby configuring the state of atleast one meta-stable switch of a plurality of meta-stable switchesdisposed within the dielectric solution and at least one synapsethereof. U.S. Patent Publication No. 20040193558 is incorporated hereinby reference.

Another example of a Knowm™ network or device is disclosed U.S. PatentPublication No. 20050015351, entitled “Nanotechnology Neural NetworkMethods and Systems” by inventor Alex Nugent, which published on Jan.20, 2005. U.S. Patent Publication No. 20050015351 generally discloses aphysical neural network (i.e., a Knowm™ network), which constitutes aconnection network comprising a plurality of molecular conductingconnections suspended within a connection gap formed between one or moreinput electrodes and one or more output electrodes. One or moremolecular connections of the molecular conducting connections can bestrengthened or weakened according to an application of an electricfield, frequency, and the like across the connection gap.

The notion of a connection network can be applied to a nanoscalearchitecture of the cross bar array. Rather than a Knowm™ synapseforming a connection at the intersection of the cross bar, theconnection is formed by the mechanical contact of the electrodesthemselves, which are implemented as nanowires. When two wires areraised to opposing voltages, columbic attraction brings the wirestogether. Charging the wires can be used to overcome the Van der Waalsattractive force and allows the mechanical strain to bring the nanowireto its ground state. The same self-organizing principles that weredeveloped for a Knowm connection can be applied to an unreliablecross-bar electrode junction.

The notion of a connection network can be applied to a nanoscaleswitching molecule architecture. Rather than a Know™ synapse forming aconnection at the intersection of the cross bar, the connection isformed by one or more switching molecules composed of, but not limitedto, Rotoxane. In general, all that is required is a two-state system,where an impedance change can be measured between the two states. Inaddition, it is preferable that one of the states is unstable and has anon-zero probability of transition to the ground state. When two wiresare raised to opposing voltages, an electric field is generated thatswitches the molecules into the meta-stable state. The sameself-organizing principles that were developed for a Knowm connectioncan be applied to a meta-stable switching molecule architecture.

An example of meta-stable switching is disclosed in U.S. Pat. No.7,599,895 entitled, “Methodology for the Configuration and Repair ofUnreliable Switching Elements,” which issued on Oct. 6, 2009 to AlexNugent, which is incorporated herein by reference in its entirety.

As transistor densities on modern integrated electronic chips increase,there is a growing trend toward reconfigurable architectures. Ratherthan implementing application specific integrated circuits (ASIC), it ispreferred that a design be deployed on programmable logic devices. Themove in such a direction is creating a growing trend toward an IP-baseddevelopment process, where circuits are defined by their programmingroutine rather than the actual physical layout. Rather than implementinga program to run on a processor, for example, a chip may process aprogram to build the processor.

In view of the foregoing developments in thermodynamic computing and theneed for reconfigurable architectures it is believed that one solutiontoward creating such technology involves the implementation of improvedmemristor devices, systems and components.

BRIEF SUMMARY

The following summary is provided to facilitate an understanding of someof the innovative features unique to the embodiments, and is notintended to be a full description. A full appreciation of the variousaspects of the embodiments can be gained by taking the entirespecification, claims, drawings, and abstract as a whole.

It is, therefore, one aspect of the disclosed embodiments to provide fora memristor apparatus based on a collection of meta-stable switchingelements.

It as another aspect of the disclosed embodiments to provide for localinteraction mechanism capable of re-configuration of states ofnano-scale meta-stable switches.

It is another aspect of the disclosed embodiments to provide forelectronic components, which can be formed utilizing the self-assemblingprinciples disclosed herein.

It is yet another aspect of the disclosed embodiments to provide amechanism for the reconfiguration of electronic components utilizingself-assembling principles.

It is yet another aspect of the disclosed embodiments to provide acircuit-level implementation of a universal logic gate utilizing theself-assembling principles disclosed herein.

The above and other aspects can be achieved as is now described.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a graphical representation of a plasticity rule,which can be implemented in accordance with an embodiment;

FIG. 2 illustrates a Knowm-Capacitor circuit, which can be implementedin accordance with one embodiment;

FIGS. 3( a)-3(d) illustrate circuit layout configurations, which can beimplemented in accordance with one or more embodiments;

FIG. 4 illustrates evaluate and feedback phase frames, which may begenerated in accordance with one or more embodiments;

FIG. 5 illustrates output/evaluate and flip/lock phase representations,which can be implemented in accordance with a first circuit layoutconfiguration;

FIG. 6 illustrates output/evaluate and flip/lock phase representations,which can be implemented in accordance with a second circuit layoutconfiguration;

FIG. 7 illustrates output/evaluate and flip/lock phase representations,which can be implemented in accordance with a third circuit layoutconfiguration;

FIG. 8 illustrates a schematic diagram of a circuit, which can beimplemented in accordance with one embodiment;

FIG. 9 illustrates a schematic diagram of a circuit, which can beimplemented in accordance with another embodiment;

FIG. 10 illustrates a schematic diagram of a circuit, which can beimplemented in accordance with an alternative embodiment;

FIG. 11 illustrates a schematic diagram of a circuit, which can beimplemented in accordance with an embodiment;

FIG. 12 illustrates a schematic diagram of a circuit, which can beimplemented in accordance with an alternative embodiment;

FIG. 13 illustrates a block-level circuit diagram, which can beimplemented in accordance with an embodiment;

FIGS. 14 and 15 illustrate a high-level block diagram of a system forindependent component analysis, which can be implemented in accordancewith a preferred embodiment;

FIG. 16 illustrates a configuration that includes a neuron with synapseinputs, in accordance with one embodiment;

FIG. 17 illustrates a system of neurons and a logic gate, in accordancewith another embodiment;

FIG. 18 illustrates a system of neurons and logic gates, in accordancewith a further embodiment;

FIG. 19 illustrates a system of neurons and logic gates in accordancewith another embodiment;

FIG. 20 illustrates a universal logic gate system that can beimplemented in accordance with a preferred embodiment;

FIG. 21 illustrates a block diagram of a meta-stable switch, which canbe implemented in accordance with an embodiment;

FIG. 22 illustrates a top view of a cross bar architecture, which can beimplemented in accordance with one embodiment;

FIG. 23 illustrates a side view of the cross bar architecture depictedin FIG. 22 in accordance with one embodiment;

FIG. 24 illustrates a top view of a switching molecule architecture,which can be implemented in accordance with another embodiment; and

FIG. 25 illustrates a side view of the switching molecule architecturedepicted in FIG. 24, in accordance with another embodiment.

DETAILED DESCRIPTION

The particular values and configurations discussed in these non-limitingexamples can be varied and are cited merely to illustrate one or moreembodiments.

Dielectrophoresis (DEP)

When a particle is suspended in a solution and subjected to an electricfield, the electric field induces a polarization in the particle. If thefield is homogeneous, the induced dipole aligns in the direction of thefield. If the field is inhomogeneous, the particle will experience aforce. The direction of the force is determined by the dielectricproperties of the particle and suspension. If the particle is morepolarizable than the surrounding medium, the particle will feel a forcein the direction of increasing field gradient, which is termed PositiveDEP. On the other hand, negative DEP results when the medium is morepolarizable than the particle.

At low frequencies, charge accumulation at the particle/medium boundarycontributes to the induced dipole, which is referred to as theMaxwell-Wagner interfacial polarization and is a function of theparticle and medium conductivity. As the frequency is increased, thisterm of the polarization has increasingly less of an effect, as themobile charges do not have time to move an appreciable distance. For thecase of a spherical particle, the time-averaged DEP force can beprovided by equation (1) as indicated below:

$\begin{matrix}{{\overset{->}{F}}_{dep} = {2\pi \; r^{3}ɛ_{0}ɛ_{m}R\; {e\left\lbrack \frac{ɛ_{p}^{*} - ɛ_{m}^{*}}{ɛ_{p}^{*} - {2ɛ_{m}^{*}}} \right\rbrack}{\nabla E^{2}}}} & (1)\end{matrix}$

For any geometry other than a sphere or ellipsoid, calculating the DEPforce is not trivial, and the applicability of equation 1 requires theparticle radius to be small compared to the changes in the gradient ofthe energy density (∇E²).

A conducting particle in a non-conducting liquid or gel will generallyfeel an attractive force toward the direction of increasing electricenergy density. As the frequency of the applied electric field isincreased, the force transitions from an attractive to a repulsiveforce. Although it is possible to use lower frequencies to attract aparticle and higher frequencies to repel in such a way as to build andbreak nanoconnections, in the present disclosure we utilize a lowerfrequency, attractive force, to build connections and increasing entropyto break connections.

Our basic requirement, which is detailed in this disclosure, is simplythat an attractive force be applied to the particle, bringing it incontact with electrodes and bridging an electrode gap. As long as theapplication of the field gradient results in an increased probability ofconnection formation, our requirements are met. Indeed, this is the caseand has been demonstrated experimentally by a number of independentorganizations.

A cross bar architecture is one where nanotubes cross each other in agrid. There are well-defined “off” and “on” states: the “off” states arepresent where the upper and lower nanotubes at the grid intersectionsare not connected and the “on” state is present when the two are pulledtogether. Because of nanotube chemical properties, the nanotubes possesstwo stable states. The separation (off state) allows a minimum forpotential energy. There is also a minimum (on state) when the two arebrought together—this is due to the Van der Waals forces that maintainthe two nanotubes together. Thus, we can switch back and forth betweenthese states resulting in metastability.

Key challenges in this architecture include fabrication and faulttolerance, where the first challenge leads to a solution that in turnleads to the second challenge. In other words, fabrication of millionsof individual nanowires on the surface of a chip requires a method forreliably placing the nanowires on the chip. The method is likely toresult in fabrication errors, which leads to the problem of how onemight build a system out of components that are unreliable. In addition,it cannot be expected that a nano-scale switching element will behave ina reliable manner, even if it is placed perfectly during fabrication.

The Plasticity Rule

Referring now to FIG. 1, illustrated a graph 100, which demonstrates aplasticity rule, which upon one or more embodiments can be based. Graph100 generally depicts a curve 103 plotted with respect to an axis 101and an axis 102. Axis 101 generally represents ƒ(y) while axis 102represents the variable “y” (i.e., y-data). Thus, graph 100 illustratesa plot of y data versus ƒ(y). Two different states 105 and 104 are alsoindicated in graph 100, where state 105 is a “negative” state orcondition and state 104 is a “positive” state or condition. Thus, aportion 109 of curve 103 is associated with state 105, while a portion111 of curve 103 is associated with state 104. Curve 103 can begenerated based on a plasticity rule. The following plasticity rule asindicated by equation (2) below, when applied to the weights of a neuronunder the influence of a data stream, has been shown to extract theindependent components of the data stream:

w _(t+1) =w _(t) +lxf(y)  (2)

In equation (2) above, the variable l represents a small positiveconstant (learning rate), while ƒ(y) represents a non-linear function ofthe total activation of the neuron, y={right arrow over (W)}{right arrowover (x)}, where {right arrow over (W)} is a weight vector and {rightarrow over (x)} an input vector. In the implementation of this rule in aphysical neural network, {right arrow over (W)} represents a matrix ofKnowm™ synapses, {right arrow over (x)} is applied via voltage pulses,and y is the total activation from the synapses. The variable l, thelearning rate, is controlled by a combination of solution viscosity,temperature and operating voltage, and possibly other factors effectingthe physical movement of nanoparticles in a liquid or gel solution.

There are many mathematical rules that have been found to extractindependent components. For a physical embodiment, it is necessary tofind a rule that can be emulated through device physics and/or particleinteractions. Equation (2) satisfies such criteria and in factrepresents a very natural condition that can be implemented in a varietyof media. The embodiments disclosed herein generally discuss how such arule can be mapped to physical processes and controlled electronically.To understand this mapping, it is necessary to discuss a theory used todescribe an unreliable, two-state connection, which has been termedPlastic Probability Theory (PPT). PPT is a way of describing theevolution of discrete-state connections or synapses via appliedplasticity rules.

Discrete-State Synapses

Consider a synapse composed of a discrete number of conducting channels,where each channel can be in an “on” or an “off” state. Further considera plasticity rule capable of modification to the number of conductingchannels. We will refer to a conducting channel as open and anon-conducting channel as closed.

The probability at any given time increment that a channel will go fromclosed to open is considered negligible outside the influence ofplasticity. This can be understood, in the light of a Knowm™ Connectionor a nanowire cross junction. In terms of a Knowm™ connection, theconnection is formed from a plurality of nanoparticles in a liquidsuspension. The increase in entropy provides for a high probability thatthe nanowires, which were attracted to the electrode gap viadipole-induced forces, will be moved to a location other than the gap.In addition, the probability that a connection will form outside theinfluence of an attractive force is negligible, so we may treat theprobability of channel dosing and opening as independent. In terms of ananowire cross junction, the stable ground state is in thenon-conducting state, as the mechanical strain requires the nanowire toeventually overwhelm the probabilistic quantum force of Van der Waalsinteraction.

The probability that a connection will go from conducting tonon-conducting is therefore the probability that a channel will close,or even more accurately as the probability the mechanical strain willovercome the Van der Waals interaction. It is clear that the probabilityof channel dosing is independent of the probability of channel opening,and that the probability of channel opening is solely due to plasticity,or in this case an electric field being applied to the system.

In terms of a switching molecule, a meta-stable molecular state causesthe transition to the ground state via quantum probability orthermal-induced bombardment. In the case of the switch molecule, isclear that the ground state need not be the non-conducting state. Theprobability that the molecule will transition from the meta-stable stateis due primarily to the addition of plasticity, or more accurately bythe addition of an electric field generated between the pre- andpost-synaptic electrode. In the case of the switching molecule, it ispossible that the molecule may make a transition from the meta-stable tothe ground state without the addition of plasticity. In this case, it isrequired that this transistor is simply less probable.

The probability that a channel will go from open to closed at any giventime increment is given by a function, E(T), which can be attributed torandom thermal motion or spontaneous transitions and is primarily afunction of temperature.

Given a total of No open channels, out of N total channels, the updateto the connection can be given as the difference between the plasticupdate, P, and the thermal breakdown, B. The plastic update and thethermal breakdown are dependant on the number of open channels. However,the plastic update can only act on dosed channels. In other words, achannel, once opened, can only be closed. If it is open, the probabilityof closing is given by E(T) as indicated in equations (3) below:

B=NoE(T)

P=(N−No)·H(No)  (3)

ΔNo=P−B=(N−No)·H(No)−NoE(T)

In equations (3) indicated above, H(No) represents a PlasticityProbability Field (PPF), which will be discussed shortly. The stablepoints of equations (3) occur when the plastic update equals the thermalbreakdown. Solving for the PPF we have:

$\begin{matrix}{{H({No})} = \frac{{NoE}(T)}{N - {No}}} & (4)\end{matrix}$

For a given thermal breakdown probability, E(T), equation (4) providesthe minimum Instantaneous Probability (IP) necessary to oppose thethermal breakdown. An IP less than that given by equation (4) can resultin the loss of open channels. An IP greater than equation (4) can resultin a gain of open channels. To find the stable points a specific PPFshould be selected. Consider the following PPF provided by equation (5):

$\begin{matrix}{{H({No})} = {\alpha \; {No}\; ^{\frac{- {No}^{2}}{2\sigma^{2}}}}} & (5)\end{matrix}$

For a given No, equation (5) provides the IP that a closed channel willopen. If equations (4) and (5) are graphed, their intersectionrepresents the equilibrium number of open channels, No.

Plastic Probability Fields

Consider a Knowm™ synapse. The dielectrophoretic force causes theaggregation of nanoparticles to areas of high field gradient. This leadsto nanowires bridging the electrode gap formed from pre- andpost-synaptic electrodes. If the particles are conducting, the localelectric field breaks down, which inhibits the growth of neighboringwires. This results in a set number of possible connections. Withoutthermal breakdown, i.e., the force of random thermal collisions, thewires would remain indefinitely and eventually reach the maximumpossible, N. Under the influence of thermal breakdown, however, theconnection will not reach the maximum number of channels, but insteadachieves a balance between thermal degradation and plastic updates.

In addition, consider nanowire cross junction array architecture. Theprobability that a closed channel will open is related to the differencebetween pre- and post-synaptic nanowire electrode voltages. As thedevice dimensions shrink and the nanowire supports become closer, thebistable switch becomes a meta-stable switch. FIG. 21, for example,illustrates a block diagram of a system 2100 composed of a meta-stableswitch 2104, which can be implemented in accordance with an embodiment.In the example depicted in FIG. 21, the meta-stable switch is disposedbetween at least one pre-synaptic electrode 2102 and at least onepost-synaptic electrode 2106. Indeed, as the operating voltage isreduced, the “configuration” voltage becomes comparable to the“read-out” voltage. As the read-out voltage has the potential to unlatchthe cross-junction, the act of reading the junction state increases theprobability of junction failure. As the support become closer, there isa relatively high probability that the switch will make a spontaneoustransition from the meta-stable to the stable state. When this occurs,the same principles and circuitry discussed for the Knowm™ synapses canbe applied to a cross bar architecture.

FIG. 22 illustrates a top view of a system 2200 that can implement sucha cross bar architecture in accordance with one embodiment. FIG. 23illustrates a side view of the cross bar architecture of system 2200depicted in FIG. 22 in accordance with one embodiment. In general,system 2200 generally includes a plurality of electrodes 2222, 2224,2226, 2228, and 2230 arranged in a cross bar architecture. Pre-synapticelectrodes 2224 and 2228 may potentially contact post-synapticelectrodes, configured in association with a plurality of nanowires2238, 2240 and 2242. Post-synaptic electrodes 2238, 2240 and 2242 may becontacted electrically via electrode 2232, 2234 and 2236. It can be seenfrom FIG. 22 that each post-synaptic electrode may be configured as aplurality of nanowires, where each nanowire is capable of one mechanicalstress contact, as indicated in FIG. 23.

Supports 2222, 2226 and 2230 raise nanowire 2231 above pre-synapticelectrode 2224. The application of a voltage gradient across pre- andpost-synaptic electrode leads to a non-zero probability that a nanowire2231 will transition from the OFF state to the ON state. In addition,the application of a voltage to the junction in the ON state will leadto a non-zero probability that nanowire 2231 will transition from the ONstate to the OFF state. The application of pre- and post-synapticvoltage pulses, as outlined in this disclosure, leads to a reliablesynaptic junction suitable for object recognition and universal logicfunction.

In addition, consider a switching molecule architecture. The probabilitythat a dosed channel will open is related to the difference between pre-and post-synaptic electrode voltages, as these voltage create anelectric field that may switch the molecule into a conducting state.Alternatively, the notions of closed and open are arbitrary in atwo-state system, so the application of an electric field via appliedpre- and post-synaptic voltage may cause an increase probability oftransition into a non-conducting state. When either of these twosituations occurs, the same principles and circuitry discussed for theKnowm™ synapses can be applied to the switching molecule architecture.

FIG. 24 illustrates a top view of a system 2400, which can implement aswitching molecule architecture, in accordance with another embodiment.FIG. 25 illustrates a side view of the switching molecule architecturesystem 2400 depicted in FIG. 24. Pre-synaptic electrodes 2408 and 2410,composed of an electrical conducting material, are placed perpendicular,either under or above, post-synaptic electrodes 2402, 2404 and 2406.Switching molecule junctions 2403 and 2405 are located between pre- andpost-synaptic electrodes, as indicated in FIG. 25. The application of avoltage gradient across pre- and post-synaptic electrodes leads to anon-zero probability that the plurality of switching molecules 2403 and2405 will transition from the OFF state to the ON state. In addition,the application of a voltage to the junction in the ON state will leadto a non-zero probability that the plurality of switching molecules 2403and 2405 will transition from the ON state to the OFF state. Theapplication of pre- and post-synaptic voltage pulses, as outlined inthis disclosure, leads to a reliable synaptic junction suitable forobject recognition and universal logic function.

A Plastic Probability Field (PPF) is a function that provides theprobability that a channel will move from a closed state to an openstate over a particular increment of time. The reason the PPF functionis a field instead of just a single value is that this probability can(and should) change as a function of the post-synaptic activation. Inaddition, a PPF does not obey most of the usual notions of probabilityfunctions, like normalization and a continuous derivative. The onlyfunctional requirement is that the PPF never attain a value greater than1 or less than zero.

As an example, suppose a connection is composed of N=10 channels. Attime step t, 5 of the channels are open (No=5). If the PPF is given as

${{H({No})} = {\frac{1}{10}{No}}},$

then there will be a 50% probability that each of the dosed channelswill open. In the absence of thermal break down, we would thereforeexpect about 7 or 8 channels to be open at time step 6. If theprobability of breakdown was P_(ƒ)=0.5, then on average no new channelswould form and the connection will have reached a stable state.

Implementation of Equation (2) as a Plastic Probability Field

Equation (1) includes two basic features that should be taken intoaccount in the physical implementation. First, as the magnitude of theactivation becomes large, the update to the connection becomesnegligible. Second, it is required that the update rule be capable ofre-enforcing two separate states, which can be referred to as the (+)state and (−) state, or State 1 and State 2, respectively. Examples ofsuch variations in state are illustrated in FIG. 1, with respect tostates 104 and 105. In FIG. 1, each state 104 and 105 represents oneside of zero. To aid in understanding of the embodiments disclosedherein, however, it may help to realize that positive and negative aresimply two distinct states, such as A and B or “On” and “Off”. In otherwords a connection can be treated as possessing at least two aspects: amagnitude and a state.

The form of the plasticity rule implemented herein requires amultiplication of the form: {right arrow over (W)}{right arrow over (x)}and x·ƒ(y). The input, x, can be implemented in the form of binaryvoltages, on and off or “1” and “0”. In general, On can represent anelectrode that has been raised to the supply voltage. Off can representan electrode that has been grounded. Alternately, and perhaps moreclearly these states can be referred to as (+) and (−), or, for example,simply State 1 and State 2. Likewise, y is also the result of a seriesof explicit multiplications as indicated by equation (6) below:

y=w ₁ x ₁ +w ₂ x ₂ +w ₃ x ₃ + . . . +w _(n) x _(n) ={right arrow over(W)}{right arrow over (x)}  (6)

The rules of multiplication, when dealing with numbers of complimentarystates can be represented by the following group of equations (7):

A(−B)=(−A)B=−AB

AB=BA  (7)

(−A)(−B)=AB

For a Knowm™ implementation of equation (2), each Knowm™ synapse shouldpreferably possesses a distinct state, and should effectively multiplyitself with the input state. In addition, the update to the neuronshould follow the rules of multiplication as well: If the output of theneuron falls in the (+) state, for example, then the connection needs tobe modified in such a way to enforce the connections ability totransform the current input state to the current output state. This canbe accomplished by multiplication with the input and can be provided,for example, in the form of a feedback mechanism that applies anelectric field to attract the particles to the electrode gap, bringnanowire cross junction together, or transition a molecule from anon-conducting to a conducting state, or the absence of an electricfield so increasing entropy may weaken a Knowm™ connection, mechanicalstrain may return a nanowire cross junction to its ground state, or aswitching molecule may return to its ground state via spontaneoustransition or reverse voltage bias.

For the case of a Knowm™ synapse, if under the frequency spectrum of theapplied electric fields the particle feels a positive DEP force, thenthis force is proportional to the square of the energy density, asprovided by equation (1). This can alternately be represented by a pre-and post-synaptic voltage, as indicated by equation (8) below:

∇|E| ² =∇|V _(pre) −V _(post)|²  (8)

The exact positions of every particle, as well as all of the forcesapplied to it, are not known. A computationally tractable model mustconsider time-averaged approximations. Random thermal motion seeks todisperse the particles through the solution. The application of avoltage difference will increase the probability that a particle willbridge the gap between pre- and post-synaptic electrodes. As a firstapproximation, we may treat the instantaneous probability that aconnection will form, or a conduction channel will open, as proportionalto the square of the voltage difference between pre- and post-synapticelectrodes. The accumulation of probability is proportional to theintegral of pre- and post-synaptic voltages over one evaluate/feedbackcycle, a cycle that will be discussed shortly.

As an alternate embodiment, the force responsible for nanowire crossjunction contact is given by the sum total of electromagnetic forcesbetween the two nanotubes minus the mechanical strain from nanowirebending. When the electromagnetic force, which is of course related tothe difference of pre- and post-synaptic voltage, overcomes themechanical strain, the nanowires come in contact and the channel isopened. Thus the accumulation of probability is proportional to theintegral of pre- and post-synaptic voltages over one evaluate/feedbackcycle, a cycle that will be discussed shortly.

As yet another alternate embodiment, the force responsible for thetransition of a conducting to a non-conducting, a non-conducting to aconducting state, or a ground state to a meta-stable state, is a resultof the electric field between the pre- and post-synaptic electrode,which of course is related to the difference between pre- andpost-synaptic electrode voltages. Thus the accumulation of probabilityis proportional to the integral of pre- and post-synaptic voltages overone evaluate/feedback cycle, a cycle that will now be discussed.

It can be seen that this basic process of a state transfer can occur inmany forms, and in all forms, the transition from one state to the otheris aided by the presence of an electric field, which is generated by theaccumulation or absence of charge on pre- and post-synaptic electrodes.The accumulation of probability of a channel opening is proportional tothe integral of the pre- and post-synaptic voltages over oneevaluate/feedback cycle.

Evaluate-Feedback Cycle

Consider one Knowm™ Connection, formed between pre-synaptic electrode“A” and post-synaptic electrode “B”. Given the inherently unstablenature of the connection in a liquid, we must provide a mechanism tostabilize the connection at a particular value while simultaneouslymonitoring the connection strength. Once the connection is at thedesired strength, we must either continually provide the feedbackmechanism to keep the connection stable, or else freeze the solution soas to prevent thermal breakdown. As previously discussed, theapplication of an activity-dependant plasticity rule can be utilized asa mechanism for setting or designating connection strengths or states.Such a plasticity rule, as applied to a Knowm™ connection, preferablyoperates on pre- and post-synaptic voltages.

To compute a post-synaptic activation, one must “evaluate” theconnections. That is, a particular input vector must be applied topre-synaptic electrodes and the post-synaptic electrode must integratethe individual contribution to form the post-synaptic activation. Thisactivation should produce a post-synaptic voltage. The post-synapticvoltage can then be utilized to apply the desired plasticity rule. Toaccomplish this, the evaluate phase can be separated from the feedbackphase. The evaluate and feedback phases may further be divided, so as toprovide for correct initial conditions.

We will assume initial conditions are correctly set, i.e. pre- andpost-synaptic voltages are preferably set to starting values. Theaccumulated probability over both the evaluate and feedback phase isgenerally responsible for the connection update. By separating theprocess into two phases, we acquire the two behaviors necessary for asuccessful integration of equation (2). The decreasing update as afunction of activity is provided through the “evaluate” phase while thecorrect update sign is accomplished with the feedback phase. Todemonstrate such a functionality, consider a simple Knowm-Capacitorcircuit 200, as illustrated by FIG. 2.

In general, the Knowm-Capacitor circuit 200 depicted in FIG. 2 includesa Knowm™ connection 202 disposed between a pre-synaptic input 209 andpost-synaptic output 202 thereof. Note that the Knowm-Capacitor circuit200 is connected to a capacitor 204 at the post-synaptic output 202.Capacitor 204 is in turn connected to a ground 208. The Knowm™connection 202 can be thought of as constituting an electro-kineticinduced particle chain. The configuration of circuit 200 is presentedherein for illustrative and exemplary purposes only. Because thenanowire cross junction and switching molecules have the same generalvoltage dependencies as a Knowm™ synapse, one just needs to replace theKnowm™ connection with another form of a meta-stable nanoscale switch.It can be appreciated that variations to circuit 200 can be implemented,while still maintaining the spirit and scope of the embodimentsdisclosed herein.

At a time t=0, both pre- and post-synaptic voltages are set to zero. Ifthe pre-synaptic input 209 is raised to a positive value, then thepost-synaptic voltage at the post-synaptic output 211 will begin to riseat a rate determined by the number of open channels. The more openchannels, the faster the post-synaptic voltage at the post-synapticoutput 211 will equilibrate with the pre-synaptic voltage. Recall thatthe update to the meta-stable switch, as given by a probability that aclosed conducting channel will open, is given by the accumulation ofprobability, which is related to the integral of the voltage differenceacross the pre- and post-synaptic electrode. If we only consider theevaluate phase, then it is apparent that as the meta-stable connection202 grows stronger, and the activity increases, the accumulatedprobability becomes smaller and approaches zero.

If, for instance, a series of input voltage pulses is applied at thepre-synaptic input 209, then the meta-stable switch 202 wouldequilibrate to a value proportional to total pre-synaptic activation.This could prove a valuable electronic filter. As it stands, thefeedback mechanism would not mirror the desired plasticity rule. Theconnection possesses a resistance somewhere between a minimum intrinsicresistance (maximum particles bridging gap, nanowire cross junctionsclosed, or conducting states of molecular switches) and a maximumintrinsic resistance (no particles bridging gap, no nanowire crossjunctions open, or no non-conducting molecular switch states). To builda system capable of 4-quadrant synaptic multiplication, there are 3basic electrode arrangements. In each arrangement, there also existsmore than one feedback mechanism capable of emulating the plasticityrule. We will discuss these three electrode arrangements, as well as thevarious feedback circuitry necessary to provide the required feedback.In all cases, a feedback phase is required, in addition to the evaluatephase, to insure proper connection modification. However, to understandthe feedback stage, it is necessary to discuss a two-state meta-stableswitch.

A meta-stable switch, in the configurations discussed herein, does notinherently posses two states, it is necessary to build the two statesinto the circuitry. One can understand this by the fact that ametastable switch, on its own, cannot multiply the state of an inputsignal. We can create a two-state meta-stable synapse by combining twoor more meta-stable switches. Take for instance the case of onepre-synaptic electrode and two post-synaptic electrodes, an arrangementthat can be referred to as configuration 1, which is depicted in FIG. 3(a), for example, as circuit layout 302.

FIGS. 3( a)-3(d) illustrate circuit layout configurations 302, 301, and303, which can be implemented in accordance with one or moreembodiments. Layout 302 of configuration 1 depicted in FIG. 3( a), forexample, generally includes an A circuit 325 and a B circuit 324. Anelectrode 321 is connected to B circuit 324, while electrodes 322 and323 are connected to A circuit 325. Electrode 322 can constitute apost-synaptic electrode 1 (PSE1), while electrode 323 can constitute apost-synaptic electrode 2 (PSE2).

The PSE1 322 can be arbitrarily assigned to State 1, while PSE2 323 isarbitrarily assigned to State 2. During the evaluate phase, thepost-Synaptic electrode with the higher voltage is considered the“winner” and feedback circuitry (i.e., to be discussed herein) saturatesthe voltages. We may view the meta-stable switch connecting the input tothe PSE 1 322 as C11 and the connection between the input and PSE 2 323as C12.

The pre-synaptic voltage may be used to encode the state of the input. Apositive voltage may arbitrarily be assigned to state 1 and a zerovoltage to state 2. If, during the evaluate phase, the pre-synapticinput is positive, then the synapse connecting the input to PSE1 andPSE2 (remember that each synapse is now represented by two meta-stableswitches) is considered to be positive if the connection facilitates thetransfer of an input in state 1 to a post-synaptic output in state 1.Likewise, if the input is in state 2, then the connection is consideredpositive if the connection facilitates the transfer of the post-synapticoutput to state 2. This is simply a restatement of the rules ofmultiplication, as outlined in equation 6. The following Table 1illustrates these features:

TABLE 1 Pre-Synaptic State Post-Synaptic State Connection state 1 1 1 12 2 2 2 1 2 1 2

A synapse may not necessarily facilitate the transfer of thepre-synaptic state to the post-synaptic state. In this case, thepost-synaptic state was determined by the summation of individualactivations from all other synapses. If a synapse state is in conflictwith the transfer of the pre-synaptic state to the post-synaptic state,then according to the above mentioned plasticity rule, the connectionshould be modified in a direction opposite its current state.

For electrode configuration 1 of circuit layout 302, if C12 was a strongconnection (i.e., one with many conducting channels) and C11 was weak,then the connection could be considered to be in State 2. This isbecause an input in state 1 (i.e., a positive input voltage) wouldmaximally affect PSE2, raising its voltage at a larger rate than C11could raise PSE1. Correspondingly, an input in state 2 (zero inputvoltage) would maximally affect PSE1 because PSE2 would receive astronger pull to ground. The PSE1 voltage would consequently be higher,forcing the neuron into state 1. Thus we have the case that a connectionwith C12>C11 facilitates the state transfers: 1→2 and 2→1. This isconsistence with a connection in state 2. One can demonstrate with thesame arguments that a synapse with C11>C12 is consistent with aconnection in state 1.

We may now consider the overall synaptic update as a function ofpost-synaptic activation on PSE1 and PSE2, and show that the functionalform matches that required by the above mentioned plasticity rule. Forillustrative purposes, consider the case of a synapse in state 1 underinputs from both state 1 and state 2. Note that we must consider theupdates to both C11 and C12, as it is only their relative strengths thatdetermine the sign of the connection. The update to the synapse can begiven as indicated by equation (9) below:

ΔW=N _(C)((A _(C11) −A _(E))−(A _(C12) −A _(E)))  (9)

As indicated by equation (9), the variable A_(C11) represents theaccumulation of connection formation probability on C11 and A_(E) is the(negative) accumulation due to random thermal motion or spontaneousstate transitions. Note that because a differential pair represents theconnection, the negative accumulation due to random thermal motion orspontaneous state transitions cancels out. Also note that when C11equals C12 (if we consider a neuron with only one input, otherwise theactivation is a function of all synapses), the accumulation on C11cancels the accumulation on C12 and the update is zero. Likewise, ifC11>>C12 or C12>>C11, the accumulation for C11 equals the accumulationfor C12 and again the accumulation cancels out, resulting in zerooverall update. This last statement can only be understood if oneconsiders an entire evaluate/feedback cycle. The specificcharacteristics of the feedback cycle will be discussed shortly.

One important detail should be mentioned. Although the negativeaccumulation due to random thermal motion or spontaneous statetransitions cancels out in equation (9), this does not mean that theindividual connection has not received the negative accumulation. Theaccumulation from plastic updates cancels the accumulation from randomthermal motion and/or spontaneous state transitions. Even without anexplicit plastic update, a residual accumulation is needed to keep theplastic update probability at a value sufficient to oppose thermalbreakdown. Otherwise a meta-stable switch would have very little chanceof remaining in its meta-stable state. We provide this residual force,and control it, by setting the periods of the evaluate and feedbackphase. For example, by doubling the period of the evaluate and feedbackphase, we double the probability that the switch will transition to themeta-stable state.

A similar result can also be achieved by increasing the supply voltage,while maintaining the same evaluate and feedback periods. This could beadvantageous because increasing the period will increase the time ofcomputation, at the cost of increase power dissipation. More aspects ofthe rule may be controlled electronically by varying the ratio ofevaluate and feedback periods. For example, by increasing the feedbackphase while maintaining the evaluate phase, the effective “width” of therule can be narrowed. Such an operation in turn allows the rule tobetter separate closely adjacent features in the input data space.

It can be appreciated that such electronic control over the plasticityrule is extremely beneficial. The control will allow the same chip toprocess many different types of data sets, and for the feedback dynamicsto be modified on-the-fly to account for variations in parameters suchas temperature, processing speed, and input data statistics.

The feedback phase can now be discussed in greater detail. Considerthree consecutive inputs, each in State 1, applied to a connection instate 1. Also consider an arbitrary initial synapse value such asC11=100 gΩ and C12=101 gΩ. During the application of the first inputduring the evaluation phase, PSE1 would receive a slightly highercurrent flux. This difference will be amplified over the course of theevaluate phase until the post-synaptic output is saturated in state 1,or PSE1=1 and PSE2=0. The relative difference between the current fluxon PSE1 and PSE2 determine the time required for the feedback circuitryto saturate the voltages in complimentary states. If the difference isinitially minute, it could take the entire evaluate phase. If theinitial difference is large, the voltages will saturate very quickly,with plenty of time left in the evaluate phase.

Note that in FIGS. 3( b) and 3(c), circuit layouts 301 and 303 are alsoillustrated, which represents variations to the circuit layout 302 orconfiguration 1. Circuit layout 301 thus represents a configuration 2,which is discussed in greater detail herein, while circuit layout 303represents a configuration 3. Circuit layout 301 or configuration 2generally includes a B circuit 314 and an A circuit 315. Electrodes 311and 312 are connected to B circuit 314, while an electrode 313 isconnected to A circuit 315. A meta-stable switch 380 can be formedbetween electrode 311 and 313. Similarly, a meta-stable switch can beformed between electrode 313 and electrode 312. In circuit layout 303 ofconfiguration 3, a B circuit 335 is provided along with an A circuit336.

Electrodes 331 and 332 are connected to the B circuit 335, whileelectrodes 333 and 334 are connected to A circuit 136. A meta-stableswitch 390 is generally formed between electrode 331 and electrode 334.Similarly, a meta-stable switch 392 can be formed between electrode 334and 332. Similar meta-stable switches, although not depicted in FIG. 3(c) are generally formed between electrode 331 and electrode 333 andbetween electrode 332 and electrode 333. A detailed side view 397 ofKnowm™ connection 390 is indicated by FIG. 3( d). Note that in FIGS. 3(c) and 3(d), identical or similar parts or elements are generallyindicated by identical reference numerals.

FIG. 4 illustrates evaluate and feedback phase frames 400, 402 and 404which may be generated in accordance with one or more embodiments.Frames 400, 402, 404 can also be designated as respective frames A, B,C. Returning to the foregoing example, during the evaluate phase, C12can receive a larger accumulated probability of connection formation, asdepicted by the shaded region in frame A of FIG. 4. During the feedbackphase, the pre-synaptic voltage will flip while the feedback circuitryholds PSE1 and PSE2 at their saturated values.

Over the course of the feedback phase, C11 receives a relatively largeaccumulation while C12 will receive none. When both the evaluate andfeedback phases are taken together, C11 receives a slightly largerupdate. In the next application of the input in state 1, PSE1 will again“win”, and a feedback phase will ensure a higher accumulated probabilityof connection formation on C11 than C12. This time, however, theaccumulated probability is slightly less for C11 that it was in theprevious frame. The reason is that the increase number of openedchannels lowered the resistance on the C11 connection. This causes ahigher current flux in the next evaluate cycle.

Although both connections received an update, it is only the differencethat matters. As one can see from frames A, B, and C of FIG. 4. as thepost-synaptic activation increases, the accumulation returns to a setlevel comprising equal accumulations for both C11 and C12. As indicatedthus far, during the evaluate phase, there is no direct correspondencebetween the post-synaptic voltage and the force needed to emulate theplasticity rule as discussed above. For example, as the PSE's becomemore positive, the voltage difference between the pre-synaptic electrodeand post-synaptic electrode lessens, the electric field becomes weaker,and the attractive force dissipates. What we require is indeed just theopposite: as the post-synaptic voltage rises, the voltage differencebetween the pre- and post-synaptic electrodes needs to increase, therebyproviding the positive feedback required by our plasticity rule. We willaccomplish this by separating the read-out or evaluate phase from themodification or feedback stage.

As the post-synaptic neuron becomes increasingly activated, theprobability that the connection grows larger will decrease. We havecaptured the first aspect of the above mentioned plasticity rule, i.e.,as y becomes larger, ƒ(y) must decrease to zero. Without the feedbackphase, however, the direction of connection update is incorrect. Withonly an evaluate phase, the weight that contributes to the finalpost-synaptic neural state receives a smaller update. If this were tocontinue for only a small time, all connections would acquire equalvalues. To change the direction of the update, a simple operation can beperformed, i.e., flip the pre-synaptic value and lock the post-synapticvalue.

A clock signal cycles the neural circuitry between the evaluate andfeedback stage. FIGS. 5, 6 and 7 generally outline or summarize thisconcept. Note that in FIGS. 3( a)-3(d) and FIGS. 5, 6 and 7, identicalor similar parts or elements are generally indicated by identicalreference numerals. Circuit layout 302 of configuration 1 is depicted inFIG. 5, for example, along with associated output/evaluate and flip/lockframes. Similar features are depicted in FIGS. 6 and 7 for respectivecircuit layouts 301 and 303.

During the evaluate phase, the pre-synaptic electrodes are locked intoeither State 1 or State 2. The pre-synaptic electrodes can be seen as avoltage source driving a signal representative of either State 1 orState 2. We will refer to this as the Output stage, which is thepre-synaptic portion of the Evaluate phase. While the pre-synapticcircuitry is locked in the Output stage, the post-synaptic neuralcircuitry is locked in the Evaluate stage. In other words, while thepre-synaptic neuron is outputting, the post-synaptic neuron isevaluating. During this phase, the voltages generated by the Outputphase of the pre-synaptic neurons are driving the PSE of thepost-synaptic neural circuitry.

The post-synaptic neural circuitry provides a feedback mechanism thatpositively re-enforces the voltages seen on PSE1 and PSE2. In otherwords, the circuitry forces PSE1 and PSE2 into orthogonal states: if thevoltage on PSE1 is initially larger than the voltage on PSE2, thecircuitry further accentuates this difference until PSE1 and PSE2 isfully saturated at the supply rails. The circuitry that accomplishesthis will be discussed, but is not considered a limiting aspect of thepresent Invention. Indeed, there exist many circuits capable of thistype of positive re-enforcement. At the end of the Evaluate phase, thepre-synaptic neural circuitry flips the Output values state. In otherwords, if the Output stage was State 1, at the end of the Output phase,the pre-synaptic electrodes are driven to the complimentary state, orstate 2. We refer to this as the Flip stage of the Feedback phase.

As the pre-synaptic neuron enters the Flip stage, the post-synapticelectrode enters the Lock stage. The Lock stage effectively locks thepost-synaptic voltages in the state decided during the evaluate phase.This can be accomplished through additional circuitry or simply byallowing the continued action of the feedback circuitry. One can see theimmediate outcome of this setup: the state that is decided during theoutput/evaluate phase (i.e. receives more activation) is reinforced inthe feedback phase by increasing the electric field.

A series of logic gates can accomplish the above describedOutput/Evaluate, Flip/Lock phases. Although we have thus far onlydiscussed the case of one pre-synaptic electrode and two post-synapticelectrodes, there are in fact more arrangements. We will now detailthree possible configurations, discuss the necessary feedback mechanism,and provide example circuitry. With an understanding of the basicoutput/evaluate, flip/lock phases, the other electrode arrangements,there state encodings, and the feedback circuitry they require shouldbecome clear.

The quanta of update probability acquired during the feedback phase canbe matched to exactly balance the degradation due to increasing entropyor spontaneous state transition. In other words, the probability that ananoparticle will be removed from the electrode gap by random thermalmotion, or a nanowire cross junction springs open, or a switchingmolecule undergoes a transition to its ground state, can be balanced bythe plastic probability gained from the feedback phase. This can in turnbe used as a mechanism for temperature compensation or to simply gainmore control over the circuit parameters. By varying the time periods ofboth the evaluate and feedback phases, as well a changing the supplyvoltages, one can “dial in” the correct plastic probability update.

Because the power dissipation via resistive heating from the connectionsis preferably minimal, one could control the temperature of the chipindependently. This would allow for such things as teaching the chip ata higher speed (and higher temperature), and then processing real-timedata at a slower speed (and a lower temperature)

FIG. 8 illustrates a schematic diagram of a circuit 800 forconfiguration 1 described earlier, which can be implemented inaccordance with one embodiment. Similarly, FIG. 9 illustrates aschematic diagram of a circuit 900 for configuration 2 describedearlier, which can be implemented in accordance with another embodiment.Likewise, FIG. 10 illustrates a schematic diagram of a circuit 1000 forconfiguration 3 described earlier, which can be implemented inaccordance with an alternative embodiment. Note that in FIGS. 8-9,identical or similar parts are generally indicated by identicalreference numerals.

The first configuration (i.e., circuit 800) is essentially the same asthe prior example (i.e., circuit layout 302), but is described here forcompleteness and further clarity. Configuration 1 of circuit 800generally includes one pre-synaptic electrode per neuron and twopost-synaptic electrodes per neuron. The input is applied as a voltage,where a positive voltage, V₊, encodes one state and a lower voltage, V⁻,encodes the complimentary state. The signal is transferred to thepost-synaptic electrodes as a voltage on two differential electrodes.

Circuit 800 generally includes a plurality of electrodes 831, includingfor example, electrodes X1, X2, etc. and an A circuit 802 and a Bcircuit 804. The A circuit 802 is composed of tri-state inverters 806,808, an inverter 810 and an AND logic gate 812. The B circuit 804generally includes a pass gate 814 and a voltage keeper formed frominverters 816, 818. B circuit 804 also includes an XOR logic gate 821.Note that output from A circuit 802 is connected at node M to the inputof B circuit 804. Node M is generally connected to pass gate 814 of theB circuit 804. Circuit lines 844 and 846 of B circuit 804 representopposite voltage states. Circuit lines 840 and 842 associated with Acircuit 802 also represent opposite voltage states. Note that voltageand/or circuit values placed at circuit line 848, which is input to XORlogic gate 821 can be utilized to control flip functionality. Circuitline 850 generally comprises a PSE 1 while circuit line 852 generallyconstitutes a PSE2.

The voltage on Post-Synaptic Electrode 1 (PSE1) is compared with thevoltage on Post-Synaptic Electrode 2 (PSE2). The PSE with a greatervoltage determines the state of the neuron. By flipping the pre-synapticvoltage to the opposite voltage and locking the PSE voltages, westrengthen the connections that contributed to the final neural stateand weaken (via increasing entropy or spontaneous state transition) theconnections that did not contribute. The feedback update is an “on oroff” update, lacking incremental control, but of fixed and knownquantity. By combining the accumulated probability of connectionformation over both the evaluate and feedback stage, we have succeededin designing a circuit capable of providing a feedback that mirrors theabove mentioned plasticity rule.

The circuitry to accomplish the Output/Evaluate and Flip/Lock phases isrelatively simple. Generally, there are two basic circuit blocks can bereferred to as circuit block “A” and circuit block “B” as indicatedpreviously herein. In FIG. 8, for example, circuit block “A” constitutescircuit 802 and circuit block “B” constitutes circuit 804. Both circuitblocks A and B form the post- and pre-synaptic functions of one neuron,respectively. Consequently, if a network does not contain more that oneneural layer, then both circuit blocks may not be required.

The function of circuit block A is two-fold. First, circuit block A(e.g., circuit 802) is responsible for the evaluate stage of theevaluate phase. Second, circuit block A is generally responsible for the“lock” stage of the feedback phase. In fact, only a very simple positivefeedback circuit may be required, as can be seen, for example, in theconfiguration depicted in FIG. 8. Because a neural module willeventually output on only one line, the addition of the inverter 810 onPSE2 and the AND gate 812 provides the following transfer function togenerate the voltage on node M, which is given as input to circuit blockB. Table 2 below illustrates some of these features:

TABLE 2 PSE1 PSE2 M 1 1 0 1 0 1 0 1 0 0 0 0

Circuit block A or circuit 802 depicted in FIG. 8, for example,generally includes the two tri-state inverters 806, 806, one inverter810 and one AND gate 812. When the tri-state inverters are activated,positive feedback is given to PSE1 and PSE2. As one can see, if PSE1 ishigh, then the inverter tries to bring PSE2 low, and visa versa. Whenthe inverters are inactive, their output floats and there is nofeedback. The evaluate and feedback phases can be generated byselectively activating the feedback circuit in the following mannerusing circuit lines 840 and 842. During the beginning of the evaluatephase, the feedback is shut off. this allows the voltage on PSE1 andPSE2 to build without feedback. These voltages are thus a representationof the activation as provided by the meta-stable switches synapses.After this first stage of the evaluate phase, the feedback is turned onusing, for example, circuit lines 840 and 842.

The feedback forces the voltages on PSE1 and PSE2 into complementarystates determined by their initial value set in the previous stage. Inother words, the feedback amplifies the difference between PSE1 and PSE2voltages. When the lock stage is reached, the feedback circuitry ofcircuit 802 (i.e., the A circuit) simply remains on, thus keeping thevoltages at their previous value. At the end of the lock stage, thefeedback is turned off. Note that the inverter 810 and the AND gate 812can act to transfer the two-line representation of two states to a oneline representation of two states, as provided by the logic table above(i.e., Table 2)

Circuit block B (e.g., circuit 804) provides the pre-synaptic functionof the neurons. In the first stage of the evaluate phase, the circuit804 produces as an output the input it received during the previousstage. This output is representative of the output state of the neuronand was determined during the evaluate phase by circuit block A. (e.g.,circuit 802). After the first stage of the evaluate phase, the outputremains the same. In the first stage of the feedback phase, the outputflips. This functionality can be accomplished with, for example, passgate 814, a voltage keeper formed from inverters 816, 818, and XOR gate821 as depicted in configuration 1 of FIG. 8. Note that the XOR gate 821can be replaced by its compliment, as the choice is ultimatelyarbitrary.

Because the stages of the evaluate and feedback phases are controlled bya clock input, the relative widths of the evaluate and feedback phasesmay be changed electronically “on the fly”. The sizes of the transistorsmaking up the feedback circuitry may of course be modified to providethe best balance of chip real estate and functionality. Alternately, thestrength of the feedback may be modified electronically by changingvoltage biases. All that is reported here, for sake of clarity, is thefunctionality required. One example is provided herein, but manyvariations are of course possible.

FIG. 11 illustrates a schematic diagram of a circuit 1100, which can beimplemented in accordance with an embodiment. Circuit 1100 generallyincludes a plurality of A and B circuits including tri-state inverters1102, 1104, 1106 and 1108. Circuit 1100 also includes transistors 1109,1110, 1111, 1112, and 1114, which are connected to one another via acircuit line 1131. Additionally, tri-state inverters 1102, 1104, 1106and 1108 can be connected to one another via a circuit line 1130 andcircuit line 1132.

FIG. 12 illustrates a schematic diagram of a circuit 1200, which can beimplemented in accordance with an alternative embodiment. Circuit 1200also includes a plurality of A and B circuits along with tri-stateinverters 1202, 1204, 1206, 1208, 1210, 1212, 1212, 1214 and 1216, eachconnected to circuit lines 1240 and 1242 Circuit 1200 additionallyincludes transistors 1218, 1220, 1222, 1224, and 1226, which areconnected to one another via a circuit line 1240. A circuit control line1243 can be connected to transistors 1218, 1220, 1222, 1224, and 1226.

In addition to circuit block A and B described above, two pieces ofcircuitry can be utilized which are useful for the process of initiallyacquiring the independent component states. First, we must provide forlateral inhibition, or negative feedback between adjacent neuralmodules. This can be accomplished in exactly the same manner as incircuit block A, except this feedback is between adjacent circuitmodules. The purpose of the inhibitory (e.g., negative) feedback is tokeep adjacent neurons from acquiring the same IC state. The feedbackmust posses the ability to turn on and off (e.g., see components 1130,1132 in FIG. 11) and in fact is off for most of the post-learninglifetime of the chip. As an example, a tri-state inverter may be used toprovide negative feedback from PSE1 of one neural module to the PSE 1 ofan adjacent neural module. Alternately, the feedback could be providedbetween PSE2 electrodes, or a non-inverting tri-state amplifier betweenPSE1 and PSE2 of adjacent neural modules. All that is required is thatone neuron is capable of pushing its neighbor into another state vialateral inhibition that can be turned on and off.

The second additional piece of circuitry could be as simple as onetransistor pulling either PSE1 or PSE2 to a pre-defined state (e.g.,voltage), which can be utilized to force a neuron into a known state. Inother words, this feedback would be used for a teaching signal, perhapscoming from another chip that has already acquired the states. Thisteaching signal is important for two reasons, although it is notstrictly necessary. The teaching signal can be used to train a group ofneural modules to recognize features within the data stream. Theknowledge of what constitutes an object is communicated to the chip viathe teaching signal, which is a global signal broadcast to all neuralmodules.

All the circuitry needed to provide the feedback required to emulate theplasticity rule given in equation (2) can be accomplished with theabove-mentioned circuitry. Not to loose generality, all that is requiredis circuitry capable of providing the mechanisms of synapticintegration, plastic feedback, lateral inhibition, and/or teaching. Theattraction of particles to the pre- and post-synaptic electrode gapscorrelate with an increased conductance. By providing an increasedvoltage difference to mirror a plasticity rule, the system canauto-regulate and converge to connection strengths suitable forinformation extraction.

Note that the electrode configurations 2 and 3 respectively depicted inFIGS. 9-10 are variations of the theme developed for configuration 1depicted in FIG. 8. The basic feedback stages essentially remain thesame. Configuration 2 (i.e., circuit 900) depicted in FIG. 9 forexample, is similar to configuration 1 (i.e., circuit 800) depicted inFIG. 8. In circuit 900 of FIG. 9, rather that one pre-synaptic electrodeand two post-synaptic electrodes, however, there are two pre-synapticand one post-synaptic electrode per neural module. Circuit 900 generallyincludes an A′ circuit 902 and a B′ circuit 904. The A′ circuit 902includes tri-state inverters 806 and 808, while the B′ circuit 904 iscomposed of the same components as the B circuit 804 depicted in FIG. 8,except for the addition of a inverter 908.

The post-synaptic electrode feedback circuitry of circuit 900 (i.e.,configuration 2) provides the same mechanism to saturate the voltage;however, this time a high voltage on the post-synaptic electrodeindicates State 1 (this is arbitrary) and a low voltage indicates State2. The following figure indicates circuit block A′, which provides thefeedback circuitry. As can be seen, the feedback circuitry is simply avoltage keeper circuit that can be regulated by the addition of atri-state inverter composing one or both of the inverters in the voltagekeeper formed from inverters 806 and 808. Circuit block B′ is thusidentical to that of configuration 1, with the addition of an extrainverter on the output to force two complimentary outputs instead ofjust one.

Note that lateral inhibition can be accomplished via a tri-stateinverter between adjacent post-synaptic electrodes. The teach signal islikewise accomplished by a transistor pulling a post-synaptic electrodeto ground or V_(cc) (e.g., see transistors 1109, 1110, etc., of FIG.11).

Configuration 3 or circuit 1000 depicted in FIG. 10 simply combinesaspects of both configuration 1 and configuration 2 by representing boththe neural input and output on two electrodes. A pair of input linesdefines one “input channel” formed from a PSE1 line or electrode 1002and PSE2 line or electrode 1004. These input lines are driven to apositive voltage in a complimentary way:

State 1: Input 1=Vcc

-   -   Input 2=Gnd

State 2: Input 1=Gnd

-   -   Input 2=Vcc

As with configuration 1, the process of neural integration can be viewedas a competition between Post-Synaptic Electrode 1 (PSE1) andPost-Synaptic Electrode 2 (PSE2).

Consider the case where an input channel is in state 1, so that inputline 1 (e.g., see X1 in FIG. 10) is “V_(cc)” and input line 2 (e.g., seeX1′ in FIG. 10) is “Gnd”. If we disregard the other inputs, the totalactivation of PSE1 is the result of the strength of the meta-stableswitch connecting Input 1 and PSE1, which can be referred to as C11.Correspondingly, the total activation of PSE2 is the result of thestrength of the meta-stable switch connecting Input 1 and PSE2, whichcan be referred to as C12. The neural circuitry 1000 depicted in FIG. 10thus compares the two voltages on PSE1 and PSE2. If PSE1 is larger, theneuron is forced to output voltages on its output channel (e.g., node Min FIG. 10) characteristic of state 1. Alternately, if PSE2 is larger,the neuron is forced to output voltages on its output channel (e.g.,node M in FIG. 10) characteristic of state 2.

Based on the foregoing, it can be appreciated that four Knowm™connections can allow for 4-quadrant multiplication. Listed below is theconnection label, as described above, along with the transfer functionit facilitates:

Connection 11: State 1→state 1Connection 12: State 1→state 2Connection 21: State 2→state 1Connection 22: State 2→state 2

To further explain such circuitry, it should be noted that, given aninput in either state 1 or 2, the value of the 4 meta-stable switchescan encode either a “positive” weight:

Connection 11: Strong (many open channels)Connection 12: Weak (no or few open channels)

Connection 21: Weak Connection 22: Strong

Or a “negative” weight:

Connection 11: Weak Connection 12: Strong Connection 21: StrongConnection 22: Weak

By the addition of a feedback mechanism (i.e., feedback circuitry), thefour connection values may take on a variety of values representinghighly “positive”, highly “negative”, or anywhere in between. It shouldbe noted that there exists a degeneracy in connections encoding aparticular value. This degeneracy is simply a result of four meta-stableswitches being used to emulate a two-state system. The advantages ofthis could include noise immunity via the differential input lines,which could be important for coupling at higher switching frequencies. Atwo-line representation can also provide a larger dynamic range for asignal, which may increase the noise margin. The circuitries needed toprovide the necessary feedback for circuit module A is identical tocircuit block A in configuration 1. Likewise, the circuitry required toimplement circuit block B is identical to circuit block B inconfiguration 2.

FIG. 13 illustrates a block-level circuit 1300, which can be implementedin accordance with one embodiment. Circuit 1300 generally includes ademultiplexer 1302 and a multiplexer 1304. A plurality of control lines1306 is connected to a plurality of A and B′ circuits 1310 and also aplurality of circuit lines 1320, which in turn are connected to aplurality of A and B′ circuits 1308 and a plurality of A and B circuits1312. Demultiplexer 1302 is connected to a plurality of electrodes 1314,while the A and B′ circuits 1308 are connected to a plurality ofelectrodes 1316. Additionally, a plurality of electrodes 1318 areconnected to the A and B′ circuits 1310 and the B and A circuits 1312.Note that the multiplexer 1304 is connected to a circuit line 1303,while the demultiplexer 1302 is connected to a circuit line 1301.

Meta-stable switches can be placed at the intersections of, for example,the B′ and A electrodes, which are patterned on the surface of the chip.In this example, data can be streamed into the demultiplexer and appliedas input to one or more electrodes. If the data is streamed so as tooutput the compliment input vector (i.e. to achieve the flip function),then a B circuit is not required. Signals are integrated on the Aelectrodes of circuit module group 1308. The output of these modules isthen applied to the B′ electrodes. The signal is integrated via the Aelectrodes on circuit module group 1310, where the pattern can berepeated for additional layers. The output state of a neural modulegroup can be multiplexed and sent out on an output line 1303. The statesof the neural circuit modules within a group can be used to determinethe presence of a feature in a data stream.

FIGS. 14 and 15 illustrate a high-level block diagram of a system 1400for independent component analysis, which can be implemented inaccordance with a preferred embodiment. In general, system 1400 includesa feedback mechanism 1406 and an electro-induced state transition, suchas, for example, a Knowm™ connection of a Knowm™ network, a crossjunction of a cross bar architecture, or a switching molecule of aswitching molecule architecture. The feedback mechanism 1406 can beimplemented in the context of, for example, the feedback circuitryillustrated herein and which operates based on the plasticity ruledescribed herein. The meta-stable switch 14306 interacts with thefeedback mechanism 1406 in order to extract independent components froma data set as depicted in FIG. 14 by arrow 1402. ICA output datagenerated from system 1400 is indicated in FIG. 14 by arrow 1408. FIG.14 represents one embodiment while FIG. 15 represents anotherembodiment. The embodiment depicted in FIG. 15 shows the addition of aneural module 1405.

Based on the foregoing, it can be appreciated that the systems andmethods disclosed herein is a new technology that extracts informationfrom a data-stream. The information processed drives a state transitionin a meta-stable switch into the meta state, and random thermal motionor spontaneous state transition to switch into the ground state, ofmeta-stable switches. The statistical regularities from the data streamare coupled to state transitions of conduction of channels between pre-and post-synaptic electrodes, which modifies the number of conductingchannels and in turn drives modular integrated circuits. As indicatedherein, a group of these circuits can be made to extract thestatistically independent components of a data stream. By processinginformation, such a network, for example, remains stable in the face ofrandom thermal motion or spontaneous state transition and activelyre-configures its connections to changing conditions.

A meta-stable switch is a nano-scale switchable electrical connectionformed from nanoparticles or molecules. The meta-stable switches formsets of connections, where the group is referred to as a synapses. Thesesynapses may be modified by a plasticity rule. We must provide a way totransfer voltages produced by neural circuit modules to a force thatalters the conducting state of the meta-stable switched. Electric fieldsare the most convenient way of generating this force.

Generally speaking, modern electronics contain two components:transistors and the wires that connect them. The transistors are like amathematical function. They have an input and an output. By arrangingand building transistors in clever ways, they can be made to storeinformation. In almost all cases, modern electronics separatecomputation from memory.

Neural networks, such as a brain, also generally contain two components:neurons and the connections between them. The neurons are not unliketransistors. They too are like a mathematical function. The connectionsbetween neurons, i.e. synapses, are very different than the wiresbetween transistors. Synapses can change, which means they have amemory, and the way they change is governed by a plasticity rule.

The rule(s) is (are) simple. The rule takes as its input localinformation, and provides as its output the change in synapse strength.Knowm™ plasticity rules use two signals: the pre-synaptic signal and thepost-synaptic signal. These signals are provided as voltages onelectrodes.

Plasticity rules are capable of computation and adaptation. A Knowm™network utilizes plasticity rules to accomplish most of what it does.The rule configures groups of meta-stable switches to form reliableconnections. The rule uses the information from a data stream tospontaneously set groups of meta-stable switched at strengths thatoptimally extract information. The rule, using statistical regularitiesin the input data stream, re-configures the states of flippedmeta-stable switches. If neural circuitry is damaged and becomesunreliable or unresponsive, the rule re-wires the network to optimizeperformance by re-configuring the states of the meta-stable switches,assuming a degree of redundancy.

Neural modules, built from CMOS (or equivalent transistor-basedtechnology), can be utilized to provide feedback to pre- andpost-synaptic electrodes. This feedback creates the electric force thatmirrors a plasticity rule capable of the above mentioned feats. Theneural modules generally contain less than 40 transistors. A group ofthese modules can be used to isolate statistical regularities in a datastream. With today's technology, hundreds of thousands of these modulegroups can be built on a single integrated circuit, along with billionsof self-assembling meta-stable connections.

A meta-stable switch, by definition, is not stable. The statisticalproperties of a group of meta-stable switches, under the influence of aplasticity rule, is stable.

Implementation of Methodology in a Universal Reconfigurable Logic Gate

As transistor densities on modern integrated electronic chips increase,there is a growing trend toward reconfigurable architectures. Ratherthan implementing application specific integrated circuits (ASIC), adesign is deployed on programmable logic devices. The move is creating agrowing trend toward an IP-based development process, where circuits aredefined by their programming routine rather than the actual physicallayout. Rather than implementing a program to run on a processor, forexample, a chip can run a program to build the processor.

There are many ways to build such a system. One feature for constructinga programmable logic device is referred to as the grain size. As thedevice must be programmed, the question naturally arises as to what,exactly, is being programmed. Given a particular computational task, thedevice must use what resources are at is disposal to implement asolution. A course-grained architecture may implement a relatively smallnumber of complex modules, where each module contains an array ofvarious logic, memory, flip-flops and perhaps even entiremicroprocessors. As the architecture becomes finer, the complexity ofthe individual cell decreases and the number of the cells increase.

Perhaps the finest-grain architecture one might imagine is a block thatcan be programmed to implement any 2-input, 1-output logic gate. Byconstructing a vast array of Universal Logic Gates, one can envision asystem that can be programmed at a very fine scale, improving theultimate efficiency of the final circuit.

A hybrid CMOS/Knowm™ logic device is disclosed herein that can be“taught” to implement any of the 16 possible 2-input/i-output logicfunctions. It can be appreciated that CMOS is one example of atransistor-based technology and others may be used. Alternate logicdevices are natural extensions of this base concept. The descriptionscontained herein, although specific to a 2-input, 1-output logic device,can be extended to arbitrary input-output dimensions. The design iscomposed of a CMOS core of about 40 transistors, as well as ameta-stable switch synapse matrix formed above the CMOS core. The designis relatively space-efficient, considering the power it has to implementany of the 16 total 2-input, 1-out logic functions. An understanding ofthe process requires an understanding of the plasticity rule describedherein under the application of binary inputs, or more specificallyknowledge of the possible fixed-points or attractor states. Such aplasticity rule has also been discussed previously.

Consider the configuration of FIG. 16, which illustrates a simple system1600 that includes a neuron 1602 with two inputs 1604 and 1606,respectively connected via two synapses 1608 and 1610. The neuron 1602may only output one of two states, but achieves a graded activation,defined by equation 1612 as y=w₁x₁+w₂x₂, where w_(i) is the weightconnecting the i^(th) input to the neuron 1602. The output 1614 fromneuron 1602 can be provided as f(y)=sign(y). The state of the neuron1602 can be seen as the sign (or state) of the activation.

The values of the synapses 1608 and 1610 can be allowed to evolve underthe auspices of the AHAH plasticity rule. There are many potentialtechniques for implementing the AHAH rule as a mathematical equation.The most general description of the feedback can be simply thefollowing: “The connection between pre-synaptic electrode A andpost-synaptic electrode B is modified in the direction that facilitatesthe transfer of electrode A's state to electrode B's state.”

For a digital application with two inputs, there are only four possibleinput patterns. For the moment, will refer to the states as “+1” and“−1” rather than “1” and “0”. This is because the explicit sign (orstate) of the input is important for AHAH modification. The fourpossible input patterns are:

-   -   [+1,+1], [+1,−1], [−1,+1], [−1,−1].        -   Dataset 1

It can be verified that the stable weight vectors resulting from theapplication of these four input patterns under the AHAH plasticity ruleare provided as: [w₁, w₂]=[0,+1],[0,−1],[+1,0],[−1,0].

Such states can be referred to as S1 though S4, respectively. Thefollowing logic tables indicate the output of a neuron in each of thefour possible states. Note that S1=X2, S2=˜X2, S3=X1 and S4=˜X1, where“˜” indicates the logical compliment.

TABLE 3 S1 S2 S3 S4 [0, +1] [0, −1] [+1, 0] [−1, 0] X1 X2 S1 X1 X2 S2 X1X2 S3 X1 X2 S4 1 1 1 1 1 −1 1 1 1 1 1 −1 1 −1 −1 1 −1 1 1 −1 1 1 −1 −1−1 1 1 −1 1 −1 −1 1 −1 −1 1 1 −1 −1 −1 −1 −1 1 −1 −1 −1 −1 −1 1

One can view the output of a neuron under the influence of the AHAH ruleand processing the dataset 1, as either passing or inverting one of thetwo inputs, depending on its state. To aid in all future discussion, wewill make the substitution −1→0 to conform to standard convention. It isimportant, however, to view “0” as a state, rather than a number, sincemultiplication by zero is zero and therefore not representative of theAHAH rule. To achieve useful logic functions, we can take two neurons,each occupying a state, and NAND their outputs.

FIG. 17 illustrates a system 1700 that generally includes a logic gate1702 having an output 1704. Logic gate 1702 receives inputs respectivelyfrom the output of neurons N1 and N2 depicted in FIG. 17. Depending onthe state of the neurons, the output will follow the rules of variouslogic functions. This can be seen in table 4, where the functionality ofcircuit 1700, shown in FIG. 17, can be tested for logic functionality.

TABLE 4 [S1, S2] [X1, X2] [1, 1] [1, 2] [1, 3] [1, 4] [2, 1] [2, 2] [2,3] [2, 4] [3, 1] [3, 2] [3, 3] [3, 4] [4, 1] [4, 2] [4, 3] [4, 4] [1, 1]0 1 0 1 1 1 1 1 0 1 0 1 1 1 1 1 [1, 0] 1 1 1 1 1 0 0 1 1 0 0 1 1 1 1 1[0, 1] 0 1 1 0 1 1 1 1 1 1 1 1 0 1 1 0 [0, 0] 1 1 1 1 1 0 1 0 1 1 1 1 10 1 0 Logic 11 1 9 3 1 6 5 2 9 5 13 1 3 2 1 4 Gate

Where the logic gates have been numbered according to the followingscheme:

TABLE 5 Logic [1, 1] [1, 0] [0, 1] [0, 0] Gate 1 1 1 1 1 1 1 1 0 2 1 1 01 3 1 1 0 0 4 1 0 1 1 5 1 0 1 0 6 1 0 0 1 7 1 0 0 0 8 0 1 1 1 9 0 1 1 010 0 1 0 1 11 0 1 0 0 12 0 0 1 1 13 0 0 1 0 14 0 0 0 1 15 0 0 0 0 16

As an example, under this numbering scheme, XOR=“10”, AND=“8” andOR=“9”. As can be seen, circuit 1700 is not capable of implementingevery logic function, since the state [S1, S2] is equivalent to [S2, S1]when “NANDed”. Such a situation can create degeneracy in logicfunctionality in that differing neural states can lead to the same logicfunction. One could replace the NAND gate with a NOR gate and achievesimilar results. In all cases, because of the degeneracy, even thoughtwo neurons are capable of occupying 16 distinct states, differentneuron states still can lead to the same logic function (degeneracy).The following table lists the following attainable logic functions forvarious “extractor logic gates”. In other words, the NAND gate incircuit 1700 could be replace with a NOR gate or an XOR gate or anyother “extractor logic gate”.

TABLE 6 Attainable Logic Extractor gate Functions NAND 1, 2, 3, 4, 5, 6,9, 11, 13 NOR 4, 8, 11, 12, 13, 14, 15, 16 XOR 1, 7, 10, 16

It is unfortunate that circuit 1700 does not attain the XOR function(Logic functions 7 and 10). To attain a greater gate functionality, thecircuitry should be modified slightly. It is certainly possible toconsider a circuit composed of three neurons and two NAND gates. Thatis, three neurons NANDed together or the output of two NANDed neuronsNANDed with a third neuron. However, this gate offers little benefitfrom circuit 1700. To achieve true universal logic function, a fourneuron implementation is preferred, which is composed of 2 instances ofcircuit 1700.

Note that in a physical implementation, each input line is preferablyrepresented on a differential electrode pair, as seen in FIGS. 3 a-3 c.The differential representation can be seen explicitly in FIG. 18, whichdepicts a system 1800 that includes logic gates 1702, 1804 and 1806.Note that in FIGS. 17-20, identical parts or elements are generallyindicated by identical reference numerals. In system 1800, X1 and X2inputs are provided to neurons N1 and N2 and to neurons N3 and N4.Output from neurons N1 and N2 is provided to logic gate 1702 whoseoutput 1704 is input to logic gate 1806. Similarly, the output fromneurons N3 and N4 is provided as input to logic gate 1804 whose output1805 is provided as input to logic gate 1806. Output from logic gate1806 is provided at output 1804.

It can be demonstrated that universal logic functionality can be foundin various subspaces in the neural states. This allows designers tosimplify the circuitry considerably so that universal logic function canbe attainable with relatively little circuit overhead. Table 7 lists thelogic gate function for all 256 possible input states. The table hasbeen ordered by logic gate.

Each neuron in the circuit 1800 can occupy four states, which we haveshown previously. There are therefore 4⁴=256 possible statecombinations. The following table lists all 256 possible stateconfigurations, as well as the over-all circuit 1800 logic function. Asone can see, multiple states lead to the same logic function. This time,however, every possible logic function can be attained.

TABLE 7 Logic S1 S2 S3 S4 Gate 1 1 2 2 1 2 2 1 1 1 3 3 4 4 1 4 4 3 3 1 11 2 3 2 1 1 3 2 2 1 1 3 3 2 1 4 3 3 2 2 3 1 1 2 3 2 1 1 2 3 3 1 1 2 3 31 4 2 3 3 4 1 2 4 1 3 3 2 1 3 2 2 3 2 2 1 3 3 2 2 3 1 3 2 2 3 3 3 2 4 33 3 3 1 2 2 3 3 3 2 2 3 3 3 2 4 3 3 3 4 2 3 4 2 3 3 3 1 2 3 3 4 1 3 2 34 1 3 3 2 4 1 3 3 3 4 2 1 3 3 4 2 3 1 3 4 2 3 3 1 4 2 3 3 3 4 3 1 2 3 43 1 3 2 4 3 1 3 3 4 3 2 1 3 4 3 2 3 1 4 3 2 3 3 4 3 3 1 2 4 3 3 1 3 4 33 2 1 4 3 3 2 3 4 3 3 3 1 4 3 3 3 2 4 3 3 3 3 4 3 3 3 4 4 3 3 4 3 4 3 43 3 4 4 3 3 3 4 1 1 2 4 5 1 1 4 2 5 1 1 4 4 5 1 3 4 4 5 2 4 1 1 5 3 1 44 5 4 2 1 1 5 4 4 1 1 5 4 4 1 3 5 4 4 3 1 5 1 1 1 1 6 1 1 1 2 6 1 1 1 36 1 1 1 4 6 1 1 2 1 6 1 1 3 1 6 1 1 3 4 6 1 1 4 1 6 1 1 4 3 6 1 2 1 1 61 3 1 1 6 1 3 1 4 6 1 3 4 1 6 1 4 1 1 6 1 4 1 3 6 1 4 3 1 6 2 1 1 1 6 31 1 1 6 3 1 1 4 6 3 1 4 1 6 3 4 1 1 6 4 1 1 1 6 4 1 1 3 6 4 1 3 1 6 4 31 1 6 1 3 2 4 7 1 3 4 2 7 2 4 1 3 7 2 4 3 1 7 3 1 2 4 7 3 1 4 2 7 4 2 13 7 4 2 3 1 7 1 2 1 3 8 1 2 3 1 8 1 3 1 2 8 1 3 1 3 8 1 3 2 1 8 1 3 3 18 1 3 3 4 8 1 3 4 3 8 2 1 1 3 8 2 1 3 1 8 3 1 1 2 8 3 1 1 3 8 3 1 2 1 83 1 3 1 8 3 1 3 4 8 3 1 4 3 8 3 4 1 3 8 3 4 3 1 8 4 3 1 3 8 4 3 3 1 8 14 2 2 9 2 2 1 4 9 2 2 4 1 9 2 2 4 4 9 2 3 4 4 9 3 2 4 4 9 4 1 2 2 9 4 42 2 9 4 4 2 3 9 4 4 3 2 9 1 4 2 3 10 1 4 3 2 10 2 3 1 4 10 2 3 4 1 10 32 1 4 10 3 2 4 1 10 4 1 2 3 10 4 1 3 2 10 1 2 2 2 11 2 1 2 2 11 2 2 1 211 2 2 2 1 11 2 2 2 2 11 2 2 2 3 11 2 2 2 4 11 2 2 3 2 11 2 2 3 4 11 2 24 2 11 2 2 4 3 11 2 3 2 2 11 2 3 2 4 11 2 3 4 2 11 2 4 2 2 11 2 4 2 3 112 4 3 2 11 3 2 2 2 11 3 2 2 4 11 3 2 4 2 11 3 4 2 2 11 4 2 2 2 11 4 2 23 11 4 2 3 2 11 4 3 2 2 11 1 2 2 3 12 1 2 3 2 12 2 1 2 3 12 2 1 3 2 12 23 1 2 12 2 3 2 1 12 2 3 2 3 12 2 3 3 2 12 2 3 3 4 12 2 3 4 3 12 3 2 1 212 3 2 2 1 12 3 2 2 3 12 3 2 3 2 12 3 2 3 4 12 3 2 4 3 12 3 4 2 3 12 3 43 2 12 4 3 2 3 12 4 3 3 2 12 1 2 4 4 13 1 4 2 4 13 1 4 4 2 13 1 4 4 4 132 1 4 4 13 2 4 1 4 13 2 4 4 1 13 2 4 4 4 13 3 4 4 4 13 4 1 2 4 13 4 1 42 13 4 1 4 4 13 4 2 1 4 13 4 2 4 1 13 4 2 4 4 13 4 3 4 4 13 4 4 1 2 13 44 1 4 13 4 4 2 1 13 4 4 2 4 13 4 4 3 4 13 4 4 4 1 13 4 4 4 2 13 4 4 4 313 4 4 4 4 13 1 2 1 4 14 1 2 4 1 14 1 4 1 2 14 1 4 1 4 14 1 4 2 1 14 1 43 4 14 1 4 4 1 14 1 4 4 3 14 2 1 1 4 14 2 1 4 1 14 3 4 1 4 14 3 4 4 1 144 1 1 2 14 4 1 1 4 14 4 1 2 1 14 4 1 3 4 14 4 1 4 1 14 4 1 4 3 14 4 3 14 14 4 3 4 1 14 1 2 2 4 15 1 2 4 2 15 2 1 2 4 15 2 1 4 2 15 2 4 1 2 15 24 2 1 15 2 4 2 4 15 2 4 3 4 15 2 4 4 2 15 2 4 4 3 15 3 4 2 4 15 3 4 4 215 4 2 1 2 15 4 2 2 1 15 4 2 2 4 15 4 2 3 4 15 4 2 4 2 15 4 2 4 3 15 4 32 4 15 4 3 4 2 15 1 2 1 2 16 1 2 2 1 16 1 2 3 4 16 1 2 4 3 16 2 1 1 2 162 1 2 1 16 2 1 3 4 16 2 1 4 3 16 3 4 1 2 16 3 4 2 1 16 3 4 3 4 16 3 4 43 16 4 3 1 2 16 4 3 2 1 16 4 3 3 4 16 4 3 4 3 16

A CIRCUIT 1800 gate may certainly be utilized to achieve areconfigurable universal logic device. By setting the neural states,Table 7 shows that any logic gate can be attained. One problem, however,is the redundancy. Four neurons, each capable of occupying 4 states,lead to 256 possible combinations. To achieve universal logic function,we only need 16 states, or two neurons. By evaluating Table 7, one canidentify a subspace of neural states where two out of the four neuronalstates do not change. In this way, we only need change the states of twoneurons. Take, for instance, the case where Neuron 1 (N1) is in State 1and Neuron 2 (N2) is in state 2. In this case, we can find the followingsubspace in Table 8:

TABLE 8 S1 S2 S3 S4 LF 1 1 2 1 6 1 1 2 2 1 1 1 2 3 2 1 1 2 4 5 1 2 2 116 1 2 2 2 11 1 2 2 3 12 1 2 2 4 15 1 3 2 1 8 1 3 2 2 3 1 3 2 3 4 1 3 24 7 1 4 2 1 14 1 4 2 2 9 1 4 2 3 10 1 4 2 4 13

Multiple subspaces can be found in table 7 that cover all logicfunctions. Table 9 shows one more example, where neuron two is in state1 and neuron 4 is in state 2.

TABLE 9 S1 S2 S3 S4 LF 1 1 2 2 1 1 1 2 3 2 1 1 4 2 5 1 1 1 2 6 2 1 2 211 2 1 3 2 12 2 1 4 2 15 2 1 1 2 16 3 1 2 2 3 3 1 3 2 4 3 1 4 2 7 3 1 12 8 4 1 2 2 9 4 1 3 2 10 4 1 4 2 13 4 1 1 2 14

Recall that the four possible states can be seen as a device functionthat either passes or inverts one of the inputs. We may use this to ouradvantage so as to illuminate the redundant circuitry. We have shown howvarious configurations can be used to implement a meta-stable switchsynapse that encodes both a state and a magnitude. For the followingexample, we will use the configuration of two pre-synaptic electrodesand one post-synaptic electrode per synaptic junction. In thisconfiguration, pre-synaptic signals are represented by differentialelectrode pairs: X1,˜X1 and X2,˜X2, where ˜ indicates the logicalcomplement.

Given the differential representation, one can see how a neural statecan be permanently emulated by a direct connection to one of the inputlines. For example, Neural State 1 is consistence with a directconnection to X2 and Neural State 2 is consistent with a directconnection to ˜X2. To take advantage of the logic subspace shown in thetables above, as well as the differential pre-synaptic electrodeconfiguration, we may simplify the circuitry as shown in system 1900 ofFIG. 19.

By application of a teaching and a teach-enable signal, it is a simplematter to initialize the neurons in the ULG into the desired states.Indeed, teaching is simply the process of forcing a neuron into apre-determined state. We may do this by selectively charging orgrounding the post-synaptic electrodes. To achieve independent controlover all neuron states within the ULG, a separate teaching could be usedfor each neuron. This would require 2 teach input lines, 1 teach enableline, as well as the two input and one output line. There are many waysto initialize the neural states. We will describe one such way as anillustration of the kind of data-stream manipulations that are possible.It should be apparent from this that there are many possibilities.

Consider a subspace where the state of neurons 2 and 4 determine thelogic function of the ULG. Further consider a DataStream composed of thedata vectors:

[1,1],[1,−1],[−1,1]t[−1,−1]

To initialize a neuron into logic function 6, for example, we wouldprovide training signals consistent with neural state 1 for both N1 andN2. If the input vectors undergo a rotation, or a series ofsubstitutions, then we can emulate another neural state. To illustratethis, consider that the output of a neuron in state 1, when subjected tothe data vectors above, will generate the following output: 1, −1, 1,−1. If we wanted to initialize N1 into state 1, but N2 into state 2,then we could present the data vector set [1,1],[1,−1],[−1,1],[−1,−1] toN1 and the data vector set [1,−1], [1,1], [−1,−1], [−1,1] to N2. In thisway, each neuron is receiving the same training signals, but the inputshave undergone a transformation so that N1 is receiving training signalsconsistent with state 1 and N2 state 2. One complete circuit diagramcapable of this can be seen in FIG. 20, which illustrates a system 2000that functions as a universal logic gate (ULG).

System 2000 generally includes two input terminals 2002 and 2004 towhich respective inputs 1 and 2 can be provided. Input 1 (i.e., input2002) and input 2 (i.e., Input 2004) can be provided as binary voltagesat input terminals 2002 and 2004, respectively. Inverters 2014 and 2018provide the inverted, or compliment, voltage signal so as to representthe inputs on a differential electrode pair, discussed in FIGS. 3 a-3 c.Meta-stable switch assemblies 2006, 2008, 2010 and 2012 provide aresistive connection to one input electrode of NAND logic gate 1702.Likewise, Meta-stable switch assemblies 2020, 2022, 2024 and 2026provide a resistive connection to one input electrode of NAND logic gate1804. Via 2016 provides a direct connection to one input electrode ofNAND 1702. Likewise, Via 2018 provides a direct connection to one inputelectrode of NAND 1804.

A tri-state voltage keeper circuit, provided by inverters 2036 and 2034,can provide a positive feedback signal capable of saturating theelectrode voltage when activated by the evaluate enable control lines.Likewise, a tri-state voltage keeper circuit, provided by inverters 2038and 2040 provide a positive feedback signal capable of saturating theelectrode voltage when activated by the evaluate enable control lines.Transistors 2030 and 2032 may provide a conducting path between teach 1and teach 2 control lines and their respective electrodes when activatedby a teach enable control line. NAND logic gates 1702, 1804 and 1806provide a logical transformation of the four input lines. Circuit 2048provides for a routing circuit capable of directing either the output ofinverter 2045 or the output of NAND 1806 to output line 2050. Circuit2048 can provide a logic bypass so as to implement a flip cycle forsecond-level logic.

To understand why the output must change while the training signal isapplied, so as to explain circuit 2048, it is necessary to understandthe flip/lock cycle, which has been previously discussed. To summarize,it is necessary for the pre-synaptic electrode to flip states if theflip/lock cycle is to properly emulate the AHAH rule. If more than oneULG are connected together, so that the output of one ULG is the inputto another, then we must insure that a configuration exists so that theoutput of the first ULG flips states. If one studies the state diagramsof Table 3 it is apparent that these states satisfy this requirement. Inother words, whatever input vector one may choose, and in whatever statethe neuron may be, if one takes the compliment vector, the output of theneuron is guaranteed to flip states. In this manner, if the output ofthe ULG is configured so that the NAND circuitry is by-passed, when theinput to the ULG is flipped the output will also flip. The importance ofthis is that a ULG connected to the first ULG will receive the flipstate, which allows the AHAH plasticity rule to be properly implementedvia the flip/lock cycle. It may also be convenient to have independentcontrol over the NAND bypass. In this case, one can control this via anindependent control line, rather than linking it to the teach enablecontrol line.

Based on the foregoing, it can be appreciated that a general methodologyfor utilizing unreliable meta-stable switches has been disclosed, whichinclude a plurality of nanoscale meta-stable switching devices eachhaving a plurality of allowable conductive states, wherein the pluralityof meta-stable switches comprise resistive connects. A plasticitymechanism is also provided, which is based on a plasticity rule forcreating stable connections from the plurality of meta-stable switchesand can be configured to implement, for example, a universalreconfigurable logic gate. The plasticity mechanism can be based, forexample, on a 2-dimensional binary input data stream, depending upondesign considerations. A circuit is also associated with the pluralityof meta-stable switches, wherein the circuit provides a logic bypassthat implements a flip-cycle for second-level logic. Additionally, anextractor logic gate is associated with the plurality meta-stableswitches, wherein the extractor logic gate provides logicfunctionalities.

It will be appreciated that variations of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be desirablycombined into many other different systems or applications. Also thatvarious presently unforeseen or unanticipated alternatives,modifications, variations or improvements therein may be subsequentlymade by those skilled in the art which are also intended to beencompassed by the following claims.

What is claimed is:
 1. A memristor apparatus, comprising: a plurality ofmeta-stable switching elements, wherein said plurality of meta-stableswitching elements are driven by an AHAH (Anti-Hebbian and Hebbian)rule; and an AHAH feedback mechanism that operates on said plurality ofmeta-stable switching elements.
 2. The memristor apparatus of claim 1wherein said AHAH rule is provided by said AHAH feedback mechanism. 3.The memristor apparatus of claim 1 wherein said AHAH feedback mechanismis employed as an adaptive synaptic weight.
 4. The memristor apparatusof claim 1 wherein said AHAH feedback mechanism extracts independentcomponents from a data stream.
 5. The memristor apparatus of claim 1wherein said memristor apparatus is employable as a discrete statesynapse.
 6. The memristor apparatus of claim 3 wherein said AHAHfeedback mechanism extracts independent components from a data stream.7. The memristor apparatus of claim 5 wherein said AHAH feedbackmechanism extracts independent components from a data stream.
 8. Thememristor apparatus of claim 2 wherein said AHAH feedback mechanism isemployed as an adaptive synaptic weight.
 9. The memristor apparatus ofclaim 2 wherein said AHAH feedback mechanism extracts independentcomponents from a data stream.
 10. The memristor apparatus of claim 2wherein said memristor apparatus is employable as a discrete statesynapse.
 11. A memristor apparatus, comprising: a plurality ofmeta-stable switching elements, wherein said plurality of meta-stableswitching elements are driven by an AHAH (Anti-Hebbian and Hebbian)rule; and an AHAH feedback mechanism that operates on said plurality ofmeta-stable switching elements and wherein said AHAH rule is provided bysaid AHAH feedback mechanism.
 12. A method of forming a memristorapparatus, said method comprising: driving a plurality of meta-stableswitching elements, wherein said plurality of meta-stable switchingelements are driven by an AHAH (Anti-Hebbian and Hebbian) rule; andoperating said plurality of meta-stable switching elements via an AHAHfeedback mechanism to comprise said memristor apparatus.
 13. The methodof claim 12 wherein said AHAH rule is provided by said AHAH feedbackmechanism.
 14. The method of claim 12 wherein said AHAH feedbackmechanism is employed as an adaptive synaptic weight.
 15. The method ofclaim 12 wherein said AHAH feedback mechanism extracts independentcomponents from a data stream.
 16. The method of claim 12 wherein saidmemristor apparatus is employable as a discrete state synapse.